Systems and methods for tracking dribbling in sporting environments

ABSTRACT

Systems and methods are provided for tracking a dribbling motion associated with a person engaged in either a training session for a sporting event or the live play of the sporting event. In sporting events, a dribbling motion can be associated with a repetitive movement or a short trajectory sequence between changes of direction of a ball, puck or other object used in the sporting event. The repetitive movement or short trajectory sequence used for dribbling can involve an up-and-down movement such as in basketball or a back-and-forth movement such as in soccer or hockey. The system can use one or more cameras to capture images of a person dribbling an object and at least one processor to analyze the images to determine and evaluate one or more characteristics associated with the dribbling motion.

BACKGROUND

Athletes often spend countless hours training in order to improve theirskill level so that they can become more competitive in sporting events,such as basketball games, soccer games, hockey games, and games of othersports. In an effort to assist athletes in improving their skill level,systems have been developed that track an athlete's performance whiletraining or playing a game and then provide feedback indicative of theperformance. Such feedback can then be evaluated for helping the athleteto improve his skill level. As an example, commonly-assigned U.S. Pat.No. 7,094,164 describes a system that tracks the trajectory of abasketball during a basketball shot so that the shooter can use feedbackfrom the system for the purpose of improving his skill at shootingbasketballs.

Tracking the performance of a dribbler, such as a basketball, soccer, orhockey dribbler, presents various challenges that may limit theeffectiveness of a tracking system that attempts to assess dribbleperformance. As an example, a dribble is often very short in durationand can occur at relatively high speeds with the ball or puck changingdirection and speed frequently. Further, the ball or puck being dribbledoften becomes at least temporarily hidden from view by players as theydribble or guard against the dribbler making it difficult toconsistently and accurately track the dribbles over time. Further,unlike some other activities, such as shooting a basketball where anideal trajectory can be characterized through certain parameters (e.g.,entry angle) that do not significantly vary from shot-to-shot, thecharacteristics of an ideal dribble can vary drastically depending onthe dribbler's situation making it difficult to accurately assess thedribbler's performance and skill level. Due to these and otherchallenges, very few attempts have been made to develop systems thatattempt to track dribble performance for training or coaching purposes.

BRIEF DESCRIPTION OF THE DRAWINGS

The included drawings are for illustrative purposes and serve only toprovide examples of possible structures and process steps for thedisclosed inventive systems and methods for providing game services toremote clients. These drawings in no way limit any changes in form anddetail that may be made to the invention by one skilled in the artwithout departing from the spirit and scope of the invention.

FIG. 1 is a block diagram of an embodiment of a dribbling motiontracking system.

FIG. 2 is a block diagram of an embodiment of a camera used in thetracking system of FIG. 1.

FIG. 3 is a block diagram of an embodiment of a computing device used inthe tracking system of FIG. 1.

FIG. 4 is an information flow diagram of an embodiment of evaluating adribbling motion during either a training sequence or a game sequence.

FIG. 5 is a block diagram of an embodiment of the object tracker of thecomputing device of FIG. 3.

FIG. 6 is an illustration of an offensive and defensive basketballplayer on an athletic playing surface.

FIG. 7 is a block diagram of an embodiment of augmented reality systemfor the tracking system.

DETAILED DESCRIPTION

Systems and methods are provided for tracking a dribbling motionassociated with a person engaged in either a training session for asporting event or the live play of the sporting event. In sportingevents, a dribbling motion can be associated with a repetitive movementor a short trajectory sequence between changes of direction of a ball,puck or other object used in the sporting event. The repetitive movementor short trajectory sequence used for dribbling can involve anup-and-down movement such as in basketball or a back-and-forth movementsuch as in soccer or hockey. The system can use one or more cameras tocapture images of a person dribbling an object and at least oneprocessor to analyze the images to determine and evaluate one or morecharacteristics associated with the dribbling motion. Thecharacteristics can be related to dribble type, dribble posture, anddribble attributes.

FIG. 1 shows an embodiment of a system 1500 for tracking dribblingmotions. For illustrative purposes, the system 1500 will be described inthe context of dribbling a basketball. However, the system can be usedwith dribbling in other sports, e.g., soccer, hockey, or field hockey,and the system may be particularly useful for tracking objects insports, such as foosball, air hockey, table tennis, etc., that involverepetitive movements or short trajectory sequences between changes ofdirection of an object used by the sport.

The system 1500 can include at least two depth sensing cameras 1502 orother types of cameras communicatively coupled to a computing device1504. The cameras 1502 can be positioned around and/or above an athleticplaying surface (such as a basketball court or other type of dribblingarea 1516) and used to capture images of a person dribbling abasketball. Various locations of the cameras 1502 are possible. As anexample, cameras 1502 may be positioned on opposite sides of thedribbling area 1516 so that the basketball is visible to at least one ofthe cameras 1502 regardless of the direction that the person is facingor which side of the person's body that the basketball is located. Atleast one camera 1502 may be mounted on a ceiling or a goal (e.g., thebackboard of a basketball goal or a pole that is coupled to thebasketball goal).

The cameras 1502 can provide the captured images to the computing device1504 as camera data. The computing device 1504 can analyze the cameradata from the cameras 1502 to track the dribbling motion of the ball anddetermine and/or evaluate one or more characteristics of the dribblingmotion, such as: the type of dribble, e.g., a cross-over dribble, abehind-the-back dribble, a between-the-legs dribble, etc.; the postureof the person performing the dribble, e.g., dribble stance, body motionand hand performing the dribbling; and the attributes of the dribbling,e.g., ball speed, dribble height, repetition rate, power, direction andturnovers.

In one embodiment, the type of dribble can refer to typical dribblingmovements associated with the game of basketball. A cross-over dribblecan refer to the movement of the ball from one hand of the player to theother hand of the player with a single dribble (or bounce) of the ballin front of the person. A behind-the-back dribble can refer to themovement of the ball from one hand of the player to the other hand ofthe player with a single dribble (or bounce) of the ball behind theperson A between-the-legs dribble can refer to the movement of the ballfrom one hand of the player to the other hand of the player with asingle dribble (or bounce) of the ball underneath the torso of theperson such that the ball travels “between the legs” of the person. Aswill be described in more detail below, the computing device 1504 candetermine the type of dribble by analyzing the motion of the ballrelative to identified body parts (e.g., torso, legs, hands, etc.) ofthe person dribbling the ball.

In another embodiment, the dribbling attributes can refer to typicalattributes associated with a dribbling motion. Ball speed can refer tothe velocity of the ball as it travels to and from a person's hand.Dribble height can refer to the distance between the person's hand andthe athletic playing surface. Repetition rate can refer to the number oftimes the ball leaves and returns to the person's hand in a defined timeperiod. Power can refer to the amount of force applied to the ball bythe person's hand. Turnovers can generally refer to a loss of control ofthe ball by the person dribbling the ball. As an example, a turnover canbe detected when the ball transfers directly from the person dribblingto a player on another team or to an out-of-bounds area.

The system 1500 can have an input device 1506 and an output device 1508communicatively coupled to the computing device 1504. The input device1506 can be any device or mechanism, e.g., a tag, that can be used toidentify the ball or the person dribbling the ball. As an example, theinput device 1506 may be worn by a player and wirelessly communicatewith the computing device 1504 information that identifies the player orprovides other information about the player. In other example, the inputdevice 1506 may be configured to receive manual inputs from the playerand wirelessly transmit to the computing device 1504 informationsubmitted by the player, such as information that identifies the playeror provides other information about the player. The identificationprocess using the input device 1506 may occur automatically during theinitialization of the system 1500 or may require one or more actions bythe person such as standing in a predefined location or performing apredetermined action. The output device 1508 can be a display screen orother similar output device that can provide the person with training orother information associated with a dribbling motion, e.g., a dribblingsequence to be repeated, and the results of the training or testingprocess.

In one embodiment, the input device 1506 and the output device 1508 areintegrated into a single apparatus, such as a smartphone or other mobiledevice. Before gameplay or training, a user may use such apparatus toinput information and thereafter to receive feedback from the computingdevice 1504 indicative of performance results or other traininginformation.

The computing device 1504 can be communicatively coupled to a lightingsystem 1510 to control the lighting effects, e.g., brightness anddirection of light, in the area where the person is dribbling the ball.In one embodiment, the lighting system 1510 can include one or morelight sources 1511. The light sources 1511 may include an incandescentbulb, a light emitting diode (LED), or a fluorescent light that isassembled into a lamp or lighting fixture. Yet other types of lightsources 1511, including light sources providing light or radiation thatis not visible to the human eye, e.g., infrared or ultraviolet lightsources, are possible in other embodiments. Depending on the type oflight source 1511 that is used, the cameras 1502 can be selected and/orconfigured to detect the light or radiation from the corresponding lightsource 1511. For example, if a light source 1511 provides infraredradiation, the camera 1502 can be equipped with an infrared sensor todetect the infrared radiation from the light source 1511.

The computing device 1504 can be used to control the lighting state,e.g., the on-state or the off-state, the lighting output apertureposition, e.g., all light can exit or a reduced quantity of light canexit, and/or the lighting output intensity, e.g., a high intensityoutput or a low intensity output, of the light sources 1511 of thelighting system 1510. In addition, the light sources 1511 may includeone or more reflectors that can be adjusted by the computing device 1504to change the direction of the light output by the light sources 1511.Further, the lighting system 1510 may include one or more mechanisms,e.g., a track and motorized trolley, for the light sources 1511 topermit the position and/or orientation of the light sources 1511 to beadjusted by the computing device 1504. The computing device 1504 may beconfigured to submit to the lighting system 1510 commands forcontrolling the states of the light sources 1511 based on an analysis ofthe images received from the cameras 1502 by the computing device 1504in an effort to achieve more optimal lighting conditions for analyzingthe captured images.

The system 1500 can also include calibration markers 1512, such as LED(light emitting diode) lights designed to emit light at a specific coloror objects that have been colored a specific color, which the computingdevice 1504 can use in calibrating (or re-calibrating) the camera datafrom the cameras 1502. The calibration markers 1512 can be identified inan image and used as a reference point that corresponds to a knownlocation. To facilitate identification of the markers 1512, the color ofthe markers can be set to a predefined color (which may be a colorrarely found in the natural environment) for which the computing device1504 searches in the images received from the cameras 1502. Once thecomputing device finds the markers 1512 in the received images, thecomputing device 1504 can then use the markers 1512 as reference pointswhen calibrating camera data from different cameras 1502. By havingknown reference points within the images, the computing device 1504 isable to identify the pixels in different sets of camera data from thecameras 1502 that are showing the same “items” from different fields ofview based on the recognition of the known reference point in thedifferent images. In one embodiment, the calibration markers 1512 can beincorporated as light sources 1511 in the lighting system 1510. In otherembodiments, other types of markers, such as court markings can be usedas known reference points.

In one example, assume that multiple cameras 1502 simultaneously capturean image of a ball that is analyzed by the computing device 1504. Usingthe markers 1512 in each of the images from the different cameras 1502,the computing device 1504 can correlate the pixel representing onephysical location in one image from one camera 1502 with a pixelrepresenting the same physical location in another image from anothercamera 1502. That is, the pixel coordinates from multiple cameras can besynchronized to a global coordinate system using the markers 1512 asreferences. Thus, the location of the ball at a given instant can beaccurately determined in space relative to the global coordinate systemusing any of the images captured by any of the cameras 1502. Therefore,as the ball moves in and out of the views of the multiple cameras (e.g.,the ball may be shielded from view of one camera 1502 while visible toanother camera 1502 as a player changes dribbles or turns his body), thelocation of the ball relative to the global coordinate system at anygiven instant can be accurately determined from the images by thecomputing device 1504 as long as the ball is in the field of view of atleast one camera 1502. Thus, over time, the location of the ball can beaccurately and consistently tracked with multiple cameras 1502 as theball comes in and out of the camera views.

One or more sensors 1514, such as accelerometers or other similar typesof sensors, can provide position, movement and/or accelerationinformation to the computing device 1504 for use in determiningdribbling motions and/or dribbling characteristics. In one embodiment,the sensors 1514 can be incorporated into the ball and/or attached to orincluded with the person dribbling the ball, and information from thesensors can be wirelessly transmitted to the computing device 1504.

The dribbling area 1516 can have one or more sensors that can provideinformation on the person and the dribbling motion to the computingdevice 1504. For example, the dribbling area 1516 can have one or moresensors embedded in the floor of the dribbling area 1516, positionedaround the perimeter of the dribbling area 1516 or otherwise associatedwith the dribbling area 1516. The sensors can include any combination ofoptical sensors, proximity sensors, infrared sensors, magnetic sensors,touch sensors, height sensors, temperature sensors, pressure sensors orany other appropriate type of sensor. The sensors used with dribblingarea 1516 can provide information on the location of the person in thedribbling area 1516 and the location and movement of the ball in thedribbling area 1516 based on the signals provided by the sensors. In oneembodiment, the dribbling area 1516 can be a bounded area to prevent theball from leaving the dribbling area 1516 while the system 1500 istracking the dribbling motion.

The computing device 1504 can be communicatively coupled to a network1518, such as a local area network (LAN) or wide area network (WAN), topermit the computing device 1504 to communicate with remote storagesystems 1520 and remote devices 1522. In one embodiment, the network1518 can be the Internet. The remote storage systems 1520 can be used toremotely store camera data, dribbling motion and characteristicinformation and other information generated and/or obtained by thecomputing device 1504. The remote device 1522 can be used to displaycamera data and/or dribbling motion and characteristic data generated orobtained by the computing device 1504. In one embodiment, the remotedevice 1522 can be a hand-held device such as a smartphone or tabletcomputer. In another embodiment, the remote device 1522 can be used inplace of the output device 1508.

The computing device 1504 can communicate wirelessly, i.e., viaelectromagnetic or acoustic waves carrying a signal, with the othercomponents of the system 1500, but it is possible for the computingdevice 1504 to communicate with the other components of the system 1500over a conductive medium (e.g., a wire), fiber, or otherwise. In oneembodiment, one or more of the system components (besides the computingdevice 1504) can communicate directly with one another (without havingto communicate with the computing device 1504) over a wireless or wiredconnection. For example, each of the cameras 1502 may communicatedirectly with one another to synchronize a start time of the cameras1502 or otherwise to synchronize the cameras 1502 or the data capturedby the cameras 1502. In another example, a camera 1502 may communicatedirectly with lighting system 1510 to change lighting conditions whenthe camera 1502 detects less than optimal lighting conditions.

In one embodiment, the cameras 1502 and the light sources 1511 of thelighting system 1510 can be stationary. However, in other embodiments,one or more of the cameras 1502 and the light sources 1511 may beportable. Each of the cameras 1502 and the light sources 1511 may bepositioned at a specific location relative to the athletic playingsurface.

One or more of the cameras 1502 may be automatically rotated or pivotedeither horizontally and/or vertically in order to adjust the field ofview of the camera 1502 without changing the location of the camera1502. Similarly, one or more of the light sources 1511 of the lightingsystem 1510 may be automatically rotated or pivoted either horizontallyand/or vertically in order to adjust the output direction of the lightsource 1511 without changing the location of the light source 1511. Inone embodiment, the rotating or pivoting of the cameras 1502 and/orlight sources 1511 may be pre-programmed into the cameras 1502 and/orlight sources 1511 such that the cameras 1502 and/or light sources 1511are rotated or pivoted according to a predetermined sequence. In anotherembodiment, the rotating or pivoting of the cameras 1502 and/or lightsources 1511 may be in response to instructions provided to the cameras1502 and/or light sources 1511 by a user, the computing device 1504, orother device.

FIG. 2 shows an embodiment of a camera 1502 that can be used with thetracking system 1500. The camera 1502 shown in FIG. 2 can include logic1530, referred to herein as “camera logic,” which may be implemented insoftware, firmware, hardware, or any combination thereof. In FIG. 2, thecamera logic 1530 is implemented in software and stored in memory 1532.However, other configurations of the camera logic 1530 are possible inother embodiments. The camera logic 1530, when implemented in software,can be stored and transported on any computer-readable medium for use byor in connection with an instruction execution apparatus that can fetchand execute instructions.

The embodiment of the camera 1502 shown in FIG. 2 can include at leastone conventional processing element 1534, which can incorporateprocessing hardware for executing instructions stored in the memory1532. As an example, the processing element 1534 may include a centralprocessing unit (CPU), a digital signal processor (DSP), a graphicprocessing unit (GPU) and/or a quantum processing unit (QPU). Theprocessing element 1534 can communicate to and drive the other elementswithin the camera 1502 via a local interface 1536, which can include atleast one bus. The camera 1502 can have a clock 1538, which can be usedto track time and synchronize operations with the other cameras 1502.

The camera 1502 can have a communication module 1540. The communicationmodule 1540 can include a radio frequency (RF) radio or other device forcommunicating wirelessly with computing device 1504 or other componentsof system 1500. The power supply 1542 has an interface that allows it toplug into or otherwise interface with an external component, such as awall outlet or battery, and receive electrical power from the externalcomponent.

As shown by FIG. 2, the camera 1502 can also include an image sensor1550, a depth sensor 1546, an audio sensor 1548 and a light sensor 1544.The image sensor 1550 can be used to record, capture or obtain images orvideos of the area surrounding or in proximity to the camera 1502. Inone embodiment, the image sensor 1550 is configured to capture twodimensional (2-D) video images of the playing area including images ofthe object being dribbled, the person performing the dribbling and anyother players in the athletic playing area. The depth sensor 1546 can beused to determine a relative distance (with respect to the depth sensor1546) of objects in the field of view of the camera 1502. The audiosensor 1548 or microphone can be used to record sound or noise occurringin the area surrounding or in proximity to the camera 1502. The lightsensor 1544 can be configured to sense ambient light in the areasurrounding the camera 1502.

The image sensor 1550 can include one or more CCDs (charge coupleddevices) and/or one or more active pixel sensors or CMOS (complementarymetal-oxide-semiconductor) sensors. The images or videos from the imagesensor 1550 can be stored as image data 1552 in memory 1532. In oneembodiment, the image data 1552 may define frames of the capturedimages. The image data 1552 can be stored in any appropriate fileformat, including, but not limited to, PNG (portable network graphics),JPEG (joint photographic experts group), TIFF (tagged image fileformat), MPEG (moving picture experts group), WMV (Windows media video),QuickTime and GIF (graphics interchange format). The sound recordingsfrom audio sensor 1548 may be incorporated into the video file from theimage sensor 1550 and stored in image data 1552. If the sound recordingfrom the audio sensor 1548 is not part of the video file, then the soundrecording can be stored in any appropriate file format, including, butnot limited to, WAV (waveform audio), MP3 (MPEG Layer III Audio), WMA(Windows media audio) and MPEG and saved in image data 1552 or elsewherein memory 1532.

In one embodiment, for each frame of image data 1552, the depth sensor1546 can provide a depth map indicating a respective depth for eachpixel of the image frame. The depth maps provided by the depth sensor1546 can be stored as depth data 1554 in memory 1532. Note that thedepth sensor 1546 may be oriented such that the distance measured by thedepth sensor 1546 is in a direction that is substantially normal to theplane of the 2D coordinate system used by the image sensor 1550,although other orientations of the depth sensor 1546 are possible inother embodiments.

From time-to-time, the camera logic 1530 can be configured to transmitthe image data 1552 and the depth data 1554 to the computing device1504. The image data 1552 and the depth data 1554 may be analyzed by thecomputing device 1504 to track the dribbling motion of the ball anddetermine one or more dribbling characteristics from the dribblingmotion. The image data 1552 and the depth data 1554 can be time-stampedbased on the time indicated by the clock 1538 in order to indicate whenthe image data 1552 and the depth data 1554 were obtained. Thus, uponreceiving the image data from multiple cameras 1502, the computingdevice 1504, based on the timestamps, can determine which image framesfrom multiple cameras 1502 were captured at substantially the same timein order to facilitate tracking of ball movement. From time-to-time, thecameras 1502 may communicate with each other and/or the computing device1504 in order synchronize their clocks so that a comparison of atimestamp for an image frame from one camera 1502 with a timestamp foran image frame of another camera 1502 accurately indicates the timedifference that the two image frames were captured. The image data 1552and the depth data 1554 may be presented to a user for analysis orreview.

Various types of image sensors 1550 and depth sensors 1546 may be usedin camera 1502. In one embodiment, the camera 1502 can be implementedusing a KINECT® camera system sold by Microsoft Corporation. In such acamera, the image sensor 1550 and depth sensor 1546 are integrated intothe same housing. The image sensor 1550 is configured to capture a videostream comprising frames of video data in which each frame is defined bya plurality of pixels. Each pixel is associated with two coordinates, anx-coordinate and a y-coordinate, representing a location in 2D space.For each frame, each pixel is assigned a color value (which may includea red component (R) value, a blue component (B) value, and a greencomponent (G) value) indicative of the color of light received by theimage sensor 1550 from the location in 2D space corresponding to thepixel's coordinates. Further, for each pixel, the depth sensor 1546measures the distance from the depth sensor 1546 to the real worldobject that is at the pixel's corresponding location in 2D space. Thedistance (which, as described above, may be in a direction substantiallynormal to the plane of the 2D coordinate system used by the image sensor1550) may be referred to as the “depth” of the corresponding pixel.Using the image data 1552 from the image sensor 1550 and the depth data1554 from the depth sensor 1546, the location of an object captured bythe image sensor 1550 can be determined in 3D space. That is, for apoint on the object, its x-coordinate and y-coordinate from the imagedata 1552 provided by the image sensor 1550 indicate its location alongtwo axes (e.g., the x-axis and y-axis), and the point's depth value fromthe depth sensor 1546, which may be referred to as the “z-coordinate,”indicates its location along a third axis (e.g., the z-axis). Notably,the coordinate system defined by the three axes is not necessarilyrelative to gravity. That is, depending on the orientation of the camera1502, gravity may be in any direction relative to the axes of thecoordinate system. Thus, unless a calibration process is performed, thedirection of gravity relative to the coordinate system may be unknown.An example of a calibration process for determining the direction ofgravity relative to the coordinate system is described by: U.S. patentapplication Ser. No. 14/874,555, entitled “Systems and Methods forMonitoring Objects in Athletic Playing Spaces” and filed on Oct. 5,2015, which is incorporated herein by reference.

In one embodiment, the depth sensor 1546 has a wave emitter (e.g., aninfrared laser projector or other type of emitter) and a wave sensor forsensing reflections of the energy emitted by the wave emitter. The waveemitter emits infrared radiation at various wavelengths into free space,although radiation at other wavelengths outside of the infrared spectrum(e.g., visible light) may be emitted in other embodiments, and the wavesensor senses the reflected energy to capture a video stream havingframes of video data. Each frame of the depth data 1554 from the sensor1546 corresponds to a respective frame of image data 1552 from the imagesensor 1550. Further, a pixel of a frame of the depth data 1554corresponds to (e.g., has the same x- and y-coordinates) and indicatesthe depth for at least one corresponding pixel in the image data 1552from image sensor 1550.

In this regard, for a frame of video data captured by the depth sensor1546, the depth sensor 1546 converts the frame to a depth map byassigning each pixel a new color value (referred to herein as “depthvalue”) representative of the pixel's depth. Thus, when the depth map isdisplayed, objects displayed as the same color within the image shouldbe approximately the same distance away from the depth sensor 1546,noting that it is often unnecessary for the depth map to actually bedisplayed during operation.

As described above, a given pixel of the image data 1552 from the imagesensor 1550 is associated with an x-coordinate and y-coordinateindicative of the pixel's location in 2D space, and the pixel isassociated with a depth value from a corresponding pixel in the depthdata 1554 provided by the depth sensor 1546 indicative of the pixel'sz-coordinate. The combination of the x-coordinate, y-coordinate, andz-coordinate defines the pixel's location in 3D space relative to thecoordinate system of the image sensor 1550. That is, the x-coordinate,y-coordinate, and z-coordinate define the location of the point fromwhich light measured for the pixel was reflected toward the image sensorfrom an object.

FIG. 3 shows an embodiment of the computing device 1504. The computingdevice 1504 may be implemented as one or more general or special-purposecomputers, such as a laptop, hand-held (e.g., smartphone), desktop, ormainframe computer. The computing device can include logic 1560,referred to herein as “device logic,” for generally controlling theoperation of the computing device 1504, including communicating with theother components of the system 1500. The computing device 1504 alsoincludes logic 1562, referred to herein as an “object tracker,” todetermine the position and movement of the object, the person handlingthe object, and any other persons in the athletic playing area andlighting system control logic 1563 to control the lighting system 1510and the light sources 1511. The computing device 1504 further includeslogic 1564, referred to herein as “computer vision logic,” forprocessing and analyzing the image data 1552 and the depth data 1554from the cameras 1502. The device logic 1560, the computer vision logic1564, lighting system control logic 1563 and the object tracker 1562 canbe implemented in software, hardware, firmware or any combinationthereof. In the computing device 1504 shown in FIG. 3, the device logic1560, the computer vision logic 1564, lighting system control logic 1563and the object tracker 1562 are implemented in software and stored inmemory 1566 of the computing device 1504. Note that the device logic1560, the computer vision logic 1564, lighting system control logic 1563and the object tracker 1562, when implemented in software, can be storedand transported on any non-transitory computer-readable medium for useby or in connection with an instruction execution apparatus that canfetch and execute instructions.

The computing device 1504 can include at least one conventionalprocessing element 1568, which has processing hardware for executinginstructions stored in memory 1566. As an example, the processingelement 1568 may include a central processing unit (CPU), a digitalsignal processor (DSP), a graphic processing unit (GPU) and/or a quantumprocessing unit (QPU). The processing element 1568 communicates to anddrives the other elements within the computing device 1504 via a localinterface 1570, which can include at least one bus. Furthermore, aninput interface 1572, for example, a keypad, keyboard or a mouse, can beused to input data from a user of the computing device 1504, and anoutput interface 1574, for example, a printer, monitor, liquid crystaldisplay (LCD), or other display apparatus, can be used to output data tothe user. In one embodiment, the input interface 1572 and the outputinterface 1574 can correspond to input device 1506 and output device1508, respectively. Further, a communication interface 1576 may be usedto exchange data among the components of the system 1500 or with network1518 as shown in FIG. 1.

As shown by FIG. 3, sensor data 1580, evaluation data 1582 and cameradata 1578 can be stored in memory 1566 at the computing device 1504. Thecamera data 1578 can include image data 1552 and depth data 1554 fromcameras 1502. The sensor data 1580 can include data and measurementsfrom sensors 1514 (e.g., accelerometers or other sensors) and/or anysensors incorporated in the dribbling area 1516. The camera data 1578,the sensor data 1580 and the evaluation data 1582 can be used and/oranalyzed by device logic 1560, computer vision logic 1564 and/or objecttracker 1562 to track the dribbling motion of an object and determineone or more characteristics of the dribbling motion.

The evaluation data 1582 can include information associated with one ormore dribbling characteristics, such as, for example, the movementsassociated with particular dribbling types. The evaluation data 1582 canalso include training information such as diagrams and videos that canbe displayed on output device 1508 to provide training instructions on“proper” dribbling form and/or technique to a user. The evaluation data1582 may include one or more testing procedures based on “proper”dribbling form that can be used to evaluate the dribbling motionassociated with a user. In one embodiment, the testing procedures can bedisplayed to the user on the output device 1508, and the object tracker1562 can evaluate a user's performance with respect to the testingprocedure in evaluation data 1582 based on the user's dribbling motioncaptured in camera data 1578.

The object tracker 1562 can receive camera data 1578, sensor data 1580,information from computer vision logic 1564 and/or other informationrelating to the ball and the person dribbling the ball to track thedribbling motion and determine one or more characteristics of thedribbling motion. Once a characteristic of the dribbling motion has beendetermined, the object tracker 1562 can compare the determined dribblingcharacteristic(s) to corresponding “proper” dribbling characteristicinformation in evaluation data 1582 to score or otherwise evaluate thedetermined dribbling characteristic(s). The “proper” dribblingcharacteristics stored in evaluation data 1582 can be preselectedparameters or techniques that are associated with a preferred dribblingmotion. In one embodiment, each determined dribbling characteristic canhave a corresponding “proper” dribbling characteristic stored inevaluation data 1582. The proper dribbling characteristic may be apredetermined number such as a predetermined speed, a predeterminednumber of dribbles per minute or a predetermined number of turnovers.The proper dribbling characteristics may also be defined relative to thebody of the person performing the dribbling, e.g., the dribble heightshould not exceed the waist of the person dribbling the ball. Further,the proper dribbling characteristics may be defined relative to actionsof the person dribbling the ball, e.g., there may be one set of properdribbling characteristics when the person is running and a different setof proper dribbling characteristics for when the person is walking orstationary. However, in other embodiments, some dribblingcharacteristics may not have a corresponding “proper” dribblingcharacteristic. In still other embodiments, the “proper” dribblingcharacteristic may be defined as a range, e.g., greater than apredetermined minimum, less than a predetermine maximum, or between apredetermined minimum and a predetermined maximum.

The computer vision logic 1564 can be used to analyze and process theimage data 1552 and depth data 1554 from the cameras 1502 stored incamera data 1578. The computer vision logic 1564 can extract informationfrom the image data 1552 and depth data 1554 in camera data 1578 usingmodels, theories and other techniques to identify or recognize theobject to be tracked and one or more participants (including the torso,arms, legs, hands, feet, etc., of the participants) involved in theathletic event associated with the object. The computer vision logic1564 can use numerous techniques to identify or recognize objects andpeople such as content-based image retrieval, pose estimation, opticalcharacter recognition, 2D code reading, shape recognition, facialrecognition, object recognition, pattern recognition and any otherappropriate identification or recognition technique. Exemplarytechniques for identifying and tracking players are disclosed in U.S.Provisional Application No. 62/297,528, entitled “Systems and Methodsfor Monitoring Objects at Sporting Events” and filed on Feb. 19, 2016,which is incorporated herein by reference.

In one embodiment, the computer vision logic 1564 can perform one ormore of the following techniques and/or processes on the image data 1552and depth data 1554 data from camera data 1578: pre-processing; featureextraction; detection/segmentation; high-level processing; and decisionmaking. The pre-processing of the camera data 1578 can involve theprocessing of the data to confirm that the data is in the proper formfor subsequent actions. Some examples of pre-processing actions caninclude noise reduction and contrast enhancement. After the camera data1578 has been pre-processed, the camera data 1578 can be reviewed oranalyzed to extract features, e.g., lines, edges, corners, points,textures and/or shapes, of various complexity from the camera data 1578.Next, in the detection/segmentation step, decisions can be maderegarding the features and/or regions that are relevant and requireadditional processing. The high-level processing of the reduced set ofcamera data 1578 (as a result of the detection/segmentation step)involves the estimation of specific parameters, e.g., object size, andclassifying of a detected object into categories. Finally, the decisionmaking step makes a determination of the identity of the detected objector person or indicates that the detected object or person is not known.

The computer vision logic 1564 can identify objects and persons that arepresent in the camera data 1578 by processing the individual images andvideos received from a camera 1502 and/or any combined or grouped imagesand videos based on camera data 1578 from multiple cameras 1502. Thecomputer vision logic 1564 can identify objects or persons using labelscarried by the objects or persons, facial recognition techniques (ifidentifying a person), profiling techniques (using the profile of theobject or person) or any other suitable recognition techniques.

In one embodiment, the object or person can have a label that isattached or affixed to the object or person and that can be recorded bycameras 1502. If the person is carrying a tag, the label can be (butdoes not have to be) incorporated into the tag carried by the person.The computer vision logic 1564 can identify the label attached to theobject or person and then identify the object or person based on storedinformation in memory 1566 correlating each label to an object orperson. In another embodiment, the computer vision logic 1564 canidentify a person using facial recognition or can identify an object ora person by using a distinguishable or identifiable profile or featureof the object or person. For example, the identification of a circularor spherical shape may indicate the presence of the ball in the frame.Similar to the process for identifying an object or person using alabel, the computer vision logic 1564 can identify facial featuresand/or other profiles or features of the object or person in the cameradata 1578 and then compare the identified facial features and/or otherprofiles or features of the asset to stored information in memory 1566correlating information on features and/or profiles to an object orperson.

The computer vision logic 1564 can send the camera data 1578 and/orinformation on the identified object or person from analyzing the cameradata 1578 to the object tracker 1562. The object tracker 1562 can useinformation on the identified object and/or persons from the computervision logic 1564 to determine a dribbling motion for the object and oneor more dribbling characteristics associated with the dribbling motion.In one embodiment, the object tracker 1562 can use synchronized andcalibrated camera data 1578 to determine a dribbling motion andcorresponding dribbling characteristics. The synchronization andcalibration of the camera data 1578 can be done by the computer visionlogic 1564 or the object tracker 1562.

The synchronization of the camera data 1578 involves ensuring that thecorresponding “frames” of camera data 1578 processed by the computervision logic 1564 or the object tracker 1562 for a give sample werecaptured substantially at the same time. In this regard, a samplegenerally refers to data from measurements that were taken substantiallyat the same time. For example, at a given instant, an image of the ballmay be captured by multiple cameras 1502. Further, the ball's positionmay be calculated from each of the images. Since the position data fromthe multiple cameras is based on image data captured substantially atthe same time in such example, the measured positions are part of thesame sample. In order to determine which frames were capturedsubstantially at the same time, a global time system may be defined. Asan example, the computing device 504 may maintain a global time systemand adjust the timestamps from each of the cameras 1502 according to theglobal time system so that the timestamps are synchronized. That is,image captured at the same time should have the same adjusted timestamp.Alternatively, the computing device 1504 (or other device maintaining aglobal time system) may from time-to-time transmit timing information tothe cameras 1504. The cameras 1504 may then use such information toadjust their respective clocks so that images having the same timestampsfrom the cameras 1504 were captured substantially at the same time.Alternatively, the computing device 1504 may analyze unsynchronizedtimestamps from the cameras 1502 and determine which frames werecaptured substantially at the same time. In such embodiment, thecomputing device 1504 may communicate with the cameras 1504 in acontrolled calibration process in order to assess timing differencesbetween the cameras 1504. As an example, each camera 1504 may report acurrent timestamp to the computing device in a handshake process so thatthe computing device 1504 can determine the camera's time relative to aglobal time system maintained by the computing device 1504 or otherwise.In other embodiments, other techniques for synchronizing the camera dataare possible.

The calibration of the camera data 1578 involves the correlation of thepixels in an image frame to a global coordinate system such that thecomputing device 1504 is aware of which pixels in different frames fromdifferent cameras 1502 represent the same physical location in space.This may be achieved, for example, by ensuring that the pixels in“frames” from different cameras 1502 representing the same physicallocation are assigned the same global coordinates. By calibrating thecamera data 1578, the object and the person dribbling the object can betracked through multiple image frames from different cameras 1502 sincethe location of the object and the person dribbling the object, asdefined in the global coordinate system, can be the same in each imageframe regardless of the field of view of the camera 1502 capturing theimage frame. Once the camera data 1578 is calibrated, the object tracker1562 can track the object through multiple image frames as the objectmoves into and out of view individual image frames. If one or morecameras 1502 become misaligned, the calibration process can be repeatedto calibrate the misaligned camera(s) 1502.

In one embodiment, the object tracker 1562 can determine a dribblingmotion by analyzing successive frames of camera data 1578 to determinechanges in the position and/or depth of the identified object and/orchanges in the position of the person preforming the dribbling motion.The object tracker 1562 can determine a dribbling motion by detecting adownward trajectory (movement away from the person) of the identifiedobject followed by a change of direction of the identified object (suchas may be caused by the object contacting the athletic playing surface)and an upward trajectory (movement toward the person) of the identifiedobject. Some exemplary techniques of calculating the trajectory of aball that may be used by object tracker 1562 can be found in U.S. Pat.No. 8,908,922 entitled “True Space Tracking of Axisymmetric ObjectFlight Using Diameter Measurement” and U.S. Pat. No. 8,948,457 entitled“True Space Tracking of Axisymmetric Object Flight Using DiameterMeasurement,” both of which patents are hereby incorporated byreference. By identifying changes associated with the upward anddownward trajectories of the object or the person dribbling the object,the object tracker 1562 can determine characteristics associated withthe dribbling motion. In one embodiment, some of the dribblingcharacteristics can be determined using conventional mathematical andphysics principles and equations based on trajectory informationextracted from the camera data 1578. The determined dribblingcharacteristics can then be stored in memory 1566 and/or scored based on“proper” dribbling characteristics stored in evaluation data 1582.

As an example, the object tracker 1562 may analyze the trajectory of theball and identify a plurality of dribbles. For one or more dribbles, theobject tracker 1562 may determine a parameter indicative of a dribblingcharacteristic, such as ball speed, dribble height, repetition rate,type of dribble, etc., and store such parameters for analysis. In somecases, the object tracker 1562 may correlate a given parameter withinformation that can be used to characterize dribbling performance. Forexample, if a given dribble is performed with the left hand, theparameter determined for the dribble may be correlated in memory with aleft hand identifier. Based on the parameters correlated with suchidentifier, the object tracker 1562 may calculate one or more scores orother statistics indicative of the player's performance with his lefthand. As an example, an average repetition rate, ball speed, or dribbleheight for the player's left hand may be calculated. If a dribble typeis identified for a particular dribble, as will be described in moredetail below, the parameter determined for the dribble may be correlatedin memory with a type identifier indicative of the dribble type. Basedon the parameters correlated with such identifier, the object tracker1562 may calculate one or more scores or other statistics indicative ofthe player's dribbling performance for the identified dribble type. If aparticular defender can be identified as guarding the player, as will bedescribed in more detail below, the parameter determined for the dribblemay be correlated in memory with an identifier that identifies thedefender. Based on parameters correlated with such identifier, theobject tracker 1562 may calculate one or more scores or other statisticsindicative of the player's dribbling performance against the defender.In other embodiments, the data can be grouped in other ways in order toprovide further insight into the player's dribbling performance relativeto certain conditions. Any of the parameters, scores, or otherstatistics described herein may be reported by the system to indicateone or more dribbling characteristics for the tracked player. Any suchparameters, scores, or other statistics may be used in order tocalculate an overall or combined assessment of the player's dribblingperformance that may be reported.

Note that techniques other than or in addition to the identification ofobjects and/or persons by computer vision logic 1564 may be used by theobject tracker 1562 to determine a dribbling motion and one or morecharacteristics associated with the dribbling motion. In one embodiment,sensor data 1580 may be analyzed by the object tracker 1562 to determinethe location and movement of an object and/or person. The sensor data1580 can then be used to determine dribbling motion and one or morecharacteristics associated with the dribbling motion.

FIG. 4 shows an embodiment of a process for evaluating the dribblingmotion of a user with the tracking system 1500 during either a trainingsequence (training mode) or a game sequence (game mode). The processbegins with a user initiating either a training sequence or a gamesequence with the tracking system 1500 being active (step 1602). If thetracking system 1500 is being used in a training mode, the user canselect a desired training sequence from the computing device 1504 usinginput device 1506. The selected training sequence can then be displayedto the user using output device 1508. The computing device 1504 canstore one or more training sequences in evaluation data 1582. Thetraining sequences can provide instruction to the user on how to executea particular dribbling motion. The training sequences can demonstratethe dribbling motion on the output device 1508 for the user to emulate.The user can then attempt to repeat the demonstrated dribbling motions,which user actions are captured by cameras 1502, during the trainingsequence. During a game sequence, the cameras 1502 may capture dribblingmotions of the user during gameplay. The camera data 1578 from thecameras 1502 can be provided to computing device 1504 and processed bythe computer vision logic 1564 to identify the object being dribbled,e.g., the ball, and the person performing the dribbling. The informationfrom the computer vision logic 1564 can then be provided to the objecttracker 1562 to identify the dribbling motion (step 1604) of the user.

The object tracker 1562 can identify the dribbling motion of the ballbased on the identification information from the computer vision logic1564, which identifies the ball, and the trajectory of the identifiedball (including any corresponding changes in depth or position of theball). Once the object tracker 1562 has identified the dribbling motion,the object tracker 1562 can identify one or more characteristics of thedribbling motion (step 1606). The object tracker 1562 can identifycharacteristics of the dribbling motion by analyzing the trajectory ofthe ball and the camera data 1578 associated with the ball and theperson performing the dribbling motion.

The object tracker 1562 can determine the hand of the person, e.g.,right hand or left hand, being used to perform the dribbling motion byidentifying the face of the person (based on facial recognition datafrom the computer vision logic 1564) and then determining the side ofthe person that is associated with the detected dribbling motion.Alternatively, the object tracker 1562 can identify the person's leftand right hands based on his body profile within the captured images.Once the object tracker 1562 has determined the hand of the personperforming the dribbling motion (the “dribble hand”), the object tracker1562 can then determine the dribbling characteristics for the dribblingmotion performed with each hand of the person.

The object tracker 1562 can use the information on the dribble hand todetermine several types of dribbling motions, e.g., cross-over dribbles,behind-the-back dribbles, or between-the-leg dribbles. The objecttracker 1562 can review information from computer vision logic 1564 anddribble hand information to determine if a particular dribbling motionhas been performed. In one embodiment, the object tracker 1562 candetermine a cross-over dribble by checking for a change in the dribblehand for the person while the ball remains in front of the person. Theobject tracker 1562 can determine a between-the-legs dribble by checkingfor a change in the dribble hand for the person while the ball travelsunderneath the torso of the person, e.g., between the legs of theperson, from the front of the person to the rear of the person. Theobject tracker 1562 can determine a behind-the-back dribble by checkingfor a change in the dribble hand for the person while the ball travelsbehind the person.

In another embodiment, the object tracker 1562 can determine one or moredribbling types based on a group of corresponding parameters that aredetermined by the object tracker 1562. Each dribbling type, e.g., a“back-to-front, between-the-legs dribble from right to left,” can bedefined as sequence or group of dribbling characteristics that caninclude a starting and/or ending dribble height, a dribble speed, adribble direction, a starting and/or ending acceleration ordeceleration, a spin on the ball, or a velocity of spin the ball. Theobject tracker 1562 can determine the particular dribblingcharacteristics occurring during a dribbling motion and then identifythe type of dribble from the dribbling characteristics. Other techniquesfor detecting the types of dribbling motions can be used in otherembodiments.

The object tracker 1562 can also determine other characteristics of thedribbling motion such as the ball speed, the dribble height, repetitionrate (e.g., dribbles per second), dribble power or other similarcharacteristics by analyzing the trajectory of the ball, i.e., thechange in the detected ball's position over subsequent frames of cameradata and the corresponding change in time that occurred betweensubsequent frames. In one embodiment, the object tracker 1562 candetermine the repetition rate by counting the number of times that theball has corresponding downward (e.g., away from the dribble hand) andupward (e.g., toward the dribble hand) trajectories associated with theuser's dribble hand within a predetermined time period. The objecttracker 1562 can determine dribble height by using the global coordinatesystem to measure the distance between the starting and ending of adownward trajectory or an upward trajectory. The object tracker 1562 candetermine ball speed by dividing the dribble height by the elapsed timefor the ball to complete either an upward or downward trajectory. Theobject tracker 1562 can determine direction by defining a perpendicularaxis with respect to a horizontal plane through the point where the ballchanges direction and then measure the angle, relative to the definedaxis, with which the ball either begins or ends an upward trajectory ora downward trajectory. The object tracker 1562 can determine dribblepower based on the ball speed for a downward trajectory and the movementof the dribble hand toward the ball prior to starting the downwardtrajectory. The object tracker 1562 can determine a turnover hasoccurred when the trajectory of the ball shows that it lands in acertain area (e.g., an out-of-bounds area) or transitions directly froma hand of the person dribbling to the hand of a player on an opposingteam, noting that players on the opposing team can be identified throughjersey color or other identification techniques. In another embodiment,the dribbling characteristics for the dribbling motion can be determinedbased on information relating to the direction of gravity andinformation relating to the location of the athletic playing surface. Instill other embodiments, still other techniques can be used to calculatethe dribbling characteristics of the dribbling motion.

The object tracker 152 can correlate each measured dribblingcharacteristic with the player's left or right hand. Thus, statisticsbased on the user's left and right hand can be determined. As anexample, the computing device 1504 can determine the user's dribblingspeed, dribbling height, turnover ratio, or other characteristics foreither his left or right hand. Thus, if desired, a player can see howhis performance dribbling with his left hand compares with hisperformance dribbling with his right hand.

Once the object tracker 1562 has determined the characteristics from thedribbling motion, the object tracker 1562 can compare the determinedcharacteristics to the preferred or “proper” characteristics (step 1608)stored in evaluation data 1582 for the training sequence. Based on thecomparison between the determined characteristics and the propercharacteristics, the object tracker 1562 can then calculate a score forthe user (step 1610). The score for the user can be based on how quicklyand/or accurately the user can reproduce the displayed training sequenceor how the user's dribbling characteristics in the game sequence compareto the proper dribbling characteristics. The scoring of the user'sperformance can be based on several accuracy factors such as how closelythe user replicates the correct sequence, the correct height for thedribbling motion and/or the correct ball placement or trajectory. Inaddition, the scoring of the user's performance in training mode canalso be based on how quickly the user is able to repeat a movement fromthe training sequence. In contrast, in game mode, additional factors,parameters or statistics, e.g., turnover rate, amount of time with theball, whether the user is guarded or unguarded, etc., may be used in thescoring of a user's performance. Once the score for the user has beencalculated the score can be displayed on the output device 1508 (step1612). In one embodiment, the score can be displayed on the outputdevice 1508 concurrent with the display of the training sequence toinform the user of how the user is performing with respect to thetraining sequence.

After the score is displayed to the user, a determination is then madeby the computing device 1504 as to whether or not the training or gamesequence has ended (step 1614). If the training sequence or gamesequence has ended, the process ends. However, if the training sequenceor game sequence has not ended, the process returns to step 1604 toidentify further dribbling motions from the user that can be evaluatedand scored. The repetition of this process can continue until thetraining sequence or game sequence has ended.

Once the training sequence or game sequence has ended, the computingdevice 1504 may recommend one or more additional training sequences forthe user based on the user's performance, e.g., score, on the completedtraining sequence or game sequence. More advanced training sequences maybe recommended if the user performed well on the completed trainingsequence or game sequence. Alternatively, one or more remedial trainingsequences may be recommended if there were particular aspects of thecompleted training sequence or game sequence in which the user did notperform well.

The computing device 1504 may also provide the user with the option toreview a completed training sequence or game sequence. The computingdevice 1504 can replay a completed training sequence or game sequence onthe output device 1508 along with a video of the user's motions duringthe training sequence or game sequence and the concurrent scorecalculations based on the user's motions. The user is then able to seethe portions of the training sequence or game sequence that the user mayhave had trouble performing.

In one embodiment, if the tracking system 1500 is used in a gamesituation, the object tracker 1562 can be used to obtain different typesof information associated with the dribbling motions occurring duringthe game. The object tracker 1562 can provide information on eachplayer's dribbling motion when guarded by a defender (e.g., a defenderis within a predetermined distance of the person with the ball and isfollowing the movements of the person with the ball) or when leftunguarded (e.g., no defender is within the predetermined distance of theperson dribbling the ball). In one embodiment, the object tracker 1562can also provide information on when the person dribbling the ball isbeing guarded by more than one defender. In another embodiment, theobject tracker 1562 can also determine whether the person dribbling theball is being tightly guarded or being loosely guarded. The objecttracker 1562 can make a determination of tight guarding if the defenderis located within a first predetermined distance range of the persondribbling the ball. If the defender is located outside of the firstpredetermined distance range (but within a second predetermined distancerange that is greater than the first predetermined distance range) theobject tracker 1562 can make a determination of loose guarding.

The object tracker 1562 can provide information on each player'sdribbling motion when guarded by a particular defender. The objecttracker 1562 can use information, such as facial recognition data, shapedata or pattern data, from the computer vision logic 1564 to identifythe defender guarding the person dribbling the ball. In one embodiment,the object tracker 1562 can identify a defender by initially determiningwhether the player has a different color or type of uniform than theperson dribbling the ball (which positive determination would make theplayer a defender). Once the object tracker 1562 determines that aplayer is a defender, the object tracker 1562 can distinguish anindividual defender from the other defenders by identifying a particularcharacteristic associated with the defender, such as by identifying auniform number of the defender through pattern recognition oridentifying the face of the defender through facial recognition.

In addition, once the object tracker 1562 has identified a defender, theobject tracker 1562 can review information from computer vision logic1564 to determine if a particular defensive movement has been performedby the defender. In one embodiment, the object tracker 1562 candetermine that a defensive movement has occurred by checking for changesin the position of the defender guarding the person dribbling the ballrelative to the movement of the ball itself. For example, the objecttracker 1562 can determine if an attempt to steal the ball is occurring(or has occurred) by checking for a movement of the defender's handtoward the position of the ball at about the same time the defender'sbody is moving toward the person dribbling the ball.

In another embodiment, the object tracker 1562 can determine one or moredefensive movements based on a group of corresponding parameters thatare determined by the object tracker 1562. Each defensive movement,e.g., a “low lunge forward to steal the ball with two hands,” can bedefined as sequence or group of defensive characteristics that caninclude hand, arm, shoulder, and leg motions of various heights, ofvarious speeds, of various directions, of various orientations, ofvarious accelerations or decelerations, with various rotations and/orwith various velocities. The object tracker 1562 can determine theparticular defensive characteristics associated with a particulardefensive movement and then identify the type of defensive movement fromthe defensive characteristics. Other techniques for detecting defensivemovements can be used in other embodiments.

The object tracker 1562 can use the information on the identifieddefender to correlate dribbling motion statistics of the persondribbling the ball to each defender guarding the person. The objecttracker 1562 can provide information on the number of times the defenderguarded the person, the cumulative amount of time the defender guardedthe person, the dribble types used against the defender, the amount orpercentage of time each hand of the person was used to dribble the ball,the number and/or percentage of turnovers, e.g., the number of times theperson dribbling the ball loses control of the ball while being guardedby the defender, and the person's dribbling attributes against thedefender. With respect to the provided information of the dribblingattributes, the object tracker 1562 can provide information on the ballspeed, dribble height, repetition rate and power of the person'sdribbling motion for each defender. The provided information on thedribbling attributes can include an average value, a range of valuesextending from a minimum value to a maximum value and/or the valueoccurring for the longest time period. In another embodiment, the objecttracker 1562 can provide similar information for when the persondribbling the ball is unguarded.

Various techniques can be used to track the performance of a playerrelative to a particular defender. As an example, by analyzing theimages captured by the cameras 1504, the object tracker 1562 canidentify the player and each defender using player identificationtechniques described in more detail above. When the player haspossession of the ball as evidenced by the ball appearing in theplayer's hands from the images or moving along a trajectory indicativeof a dribble by the player (e.g., leaving the player's hand andreturning to a hand of the player after bouncing off of the floor of thedribbling area), the object tracker 1562 may analyze the images todetermine the player's distance from each identified defender. Thedefender that is closest to the player may be identified as the defenderwho is guarding the player with the ball if he is within a predetermineddistance of the player with the ball. If desired, the defender may berequired to be within predetermined distance for at least a certain timeperiod before a guarding determination is made in order to preventdefenders who briefly pass by the player while guarding other playersfrom being mistakenly identified as guarding the player with the ball.Other techniques for determine whether a particular defender is guardingthe player with the ball are possible in other embodiments. As anexample, the body orientation of the defender may be a factor indetermining whether he or she is guarding the player with the ball. Inthis regard, a defender facing the player with the ball for an extendedtime period is likely to be guarding him. The object tracker 1562 may beconfigured to determine that a defender is guarding the player when heis facing the player within a predefined distance of the player for atleast a predefined amount of time. In another embodiment, dataindicative of the defender (e.g., a jersey number of other identifier ofthe defender) guarding the player with the ball may be manually input tothe computing device 1502 or other device of the system by a user. Yetother techniques are possible in yet other embodiments.

While an identified defender is determined to be guarding the player,the player's dribbling characteristics can be correlated with anidentifier that identifies the guarding defender. Thus, characteristicsindicative of the player's dribbling performance while being guarded bythe identified defender can be determined from the data captured by thesystem, and the object tracker 1562 can be configured to calculatevarious scores and statistics indicative of such performance. Over timeas the player is guarded by multiple defenders, his dribblingperformance against one defender can be compared to his dribblingperformance against another defender. Note that the information can beused to help train the person dribbling the ball or for other purposes,such as deciding which defender would be the most effective at guardingthe person dribbling the ball.

The object tracker 1562 can determine the impact or effect thatparticular dribbling characteristics or motions had on a defensiveplayer guarding the player dribbling the ball. For example, the objecttracker 1562 can determine if a particular dribbling characteristic wassuccessful or unsuccessful against a defensive player or whether theperson dribbling the ball was able to perform particular dribblingcharacteristics against the defensive player. A dribbling characteristiccan be considered successful if the person dribbling the ball was ableto advance past the defensive player. In contrast, a dribblingcharacteristic may be considered unsuccessful if the person dribblingthe ball was not able to advance past the defensive player or if theperson dribbling the ball committed a turnover, e.g., losing the ballout-of-bounds or losing the ball to the defensive player guarding theperson (or another defensive player). Other metrics for gauging thesuccess of a dribbling characteristic are possible in other embodiments.

FIG. 5 shows an embodiment of the object tracker 1562 that can be usedby computing device 1504. The object tracker 1562 can include ball pathlogic 1591 for generally determining the path or movement of the balland the person dribbling the ball, even when the ball and/or the personare concealed from the cameras 1502, identification logic 1592 fordetermining the location of the ball and/or the offensive and defensiveplayers or persons on the athletic playing surface, and scoring logic1595 for evaluating the dribbling motion of a person or the performanceof a defender guarding the person dribbling the ball and providing a“score” associated with the person's performance. The scoring logic 1595can evaluate a person's performance based on information frommeasurement logic 1597. Measurement logic 1597 can be used to measurethe capabilities of the person dribbling the ball and/or thecapabilities of the person(s) defending the person dribbling the ball.Improvement logic 1594 can use information from measurement logic 1597and scoring logic 1595 to determine areas where the person can improvehis/her performance. The object tracker 1562 can also include defendermotion logic 1593 to generally determine the movement and/or actions ofa defender, even when the defender is concealed from the cameras 1502,and balance logic 1599 to evaluate the balance and/or fluidity of theperson dribbling the ball and/or the person(s) defending the persondribbling the ball.

Historical data 1596 and body motion data 1598 used by the objecttracker 1562 can be stored in memory 1566 at the computing device 1504.The historical data 1596 can include information relating to previousmovements and actions of the person dribbling the ball during trainingsequences and/or live play sequences. The historical data 1596 can alsoinclude data and information on the movements and actions of thedefender(s) guarding the person dribbling the ball. Body motion data1598 can include information relating to the location and movement of aperson (both the person dribbling the ball and the defender(s)) andhis/her associated body parts, e.g., head, shoulder, elbow, hand,finger, chest, waist, back, thigh, knee, calf, hip, ankle and footduring the dribbling of the ball or defending the person dribbling theball. The body motion data 1598 can also include, where applicable, leftside and right side information and front and back informationassociated with the player's body parts.

As previously discussed, the object tracker 1562 can receive camera data1578, sensor data 1580, information from computer vision logic 1564and/or other information relating to the ball and the players or personson the athletic playing surface. The ball path logic 1591 can be used todetermine (or approximate) the path of the ball and the person dribblingthe ball during the dribbling motion even if the ball or the personcannot be identified by identification logic 1592 based on the cameradata 1578. For example, the identification logic 1592 (or the computervision logic 1564) may not be able to identify the ball or the personbecause the ball may not be present in camera data 1578. The ball maynot be present in the camera data 1578 due to the ball being concealedfrom the fields of view of the cameras 1502 by the person dribbling theball, the person(s) guarding the person dribbling the ball and/or one ormore other persons on the athletic playing surface. See e.g., FIG. 6.Further, even if the ball is present in the camera data 1578, theidentification logic 1592 may not be able to recognize the ball becausethe ball is obscured in the camera data 1578 due to poor lightingconditions, partial occlusion of the ball and/or blurring as a result ofrapid movement of the ball.

In one embodiment, the identification logic 1592 can determine if a foulor other violation occurred during the game by determining if a whistlesound occurs followed by a stoppage of play. The identification logic1592 may also determine the occurrence of a foul or other violation, byidentifying a stoppage of play followed by one or more actions of thereferee, e.g., the referee moving toward the scorer's table and makingone or more gestures. The identification logic 1592 can determine whichplayer committed the foul or violation and which type of foul orviolation occurred based on the hand and arm movements of the referee.For example, the referee may indicate a blocking foul by moving his/herhands onto his/her hips one or more times. The identification logic 1592may analyze the hand gestures for identifying a foul type (e.g.,interpret hand gestures for determining that a blocking foul hasoccurred when the referee moves his/her hands onto his/her hips within acertain time period after a whistle). The referee may also use handgestures to indicate the number of the player who committed the foul(e.g., hold up a number of fingers indicting the number), and theidentification logic 1592 may interpret such gestures to identify theplayer who committed the foul. The identification logic 1592 may also beable to determine which player committed the foul or violation and thetype of foul or violation that occurred by processing audio informationcaptured from the referee saying the player and type of foul orviolation.

The ball path logic 1591 can use the information from the identificationlogic 1592 to determine the path or trajectory of the ball. When theidentification logic 1592 is unable to identify the ball from the cameradata 1578, the ball path logic 1591 can determine an expected trajectoryor movement of the ball based on the last known location of the ballfrom identification logic 1592 and other information stored in memory1566. The ball path logic 1591 can analyze the body position of theperson dribbling the ball based on the body motion data 1598 andapproximate the expected trajectory of the ball and the time to completethe trajectory based on how the person dribbling the ball is positioned.The ball path logic 1591 can confirm (or reject) the approximatetrajectory of the ball once the identification logic 1592 is able toidentify the ball from the camera data 1578.

For example, if a person is dribbling the ball behind his back during agame, the ball may not be visible to identification logic 1592 due toocclusion from the dribbler's body as well as occlusion from otherplayers or persons on the athletic playing surface. However,identification logic 1592 may be able to detect the motion of theshoulders, arms, and hands of the dribbler and provide that informationto the ball path logic 1591. The ball path logic 1591 can then use theinformation from the identification logic 1592 and body motion data 1598to approximate the motion, trajectory, direction, spin, and velocity ofthe ball while it is not visible or detectable to identification logic1592 and predict the arrival time and location of the ball on the otherside of the dribbler when the ball becomes visible or detectable by theidentification logic 1592 from the camera data 1578.

If the ball path logic 1591 receives information from the identificationlogic 1592 that the ball is in the position expected by the ball pathlogic 1591 (subject to a margin of error), the ball path logic 1591 candetermine that the actual trajectory of the ball followed theapproximated trajectory determined by the ball path logic 1591. However,if the ball path logic 1591 receives information from the identificationlogic 1592 that the ball is in a different location than expected, theball path logic 1591 can determine that the movement of the ball did notfollow the approximated trajectory and can approximate a new trajectoryfor the ball based on the starting and ending locations for the ball. Inaddition, the ball path logic 1591 can store information in memory 1566(possibly as historical data 1596) on the starting and ending positionsof the ball, the revised approximated trajectory and the persondribbling the ball. The ball path logic 1591 can then use the storedinformation on the starting and ending position for the ball and therevised approximated trajectory when formulating an approximatedtrajectory for the ball when the ball becomes occluded in a similarsituation in the future.

In another embodiment, the ball path logic 1591 may be able to determinethe trajectory or movement of the ball even if the ball or some (or all)of the person dribbling the ball is occluded in camera data 1578. As anexample, the ball may be occluded from view, but the dribbler's elbowmay be visible. Movement of the dribbler's arm near the elbow mayindicate when the ball has reached the dribbler's hand. In this regard,a change in movement of the dribbler's arm may indicate that the ballhas reached the dribbler's hand and is being pushed downward for adribble. Further, the ball path logic 1591 may calculate the location ofthe ball at the time it is determined to reach the dribbler's hand basedon the location and orientation of the dribbler's elbow. In this regard,the arm length of the dribbler may be predetermined and used by thelogic 1591 to determine the ball's distance from the dribbler's elbow.Also, the angle of the dribbler's forearm may indicate the ball'sdirection from his elbow. By determining various locations of the ballat different times while the ball is occluded, the ball path logic 1591can estimate the trajectory of the ball between such points.

If desired, the ball path logic 1591 can use computer learning and/orartificial intelligence to establish the most likely paths the ballwould travel based on any other current data that is available (e.g.,data extracted from camera data 1578 or data from sensor data 1580 suchas depth sensor, motion sensor/accelerometer or sound information) orfrom historical data 1596 that includes information of what the personis most likely to do in a particular situation or environment. In thisregard, by analyzing the dribbler's movements over time, the ball pathlogic 1591 can learn how the dribbler likely responds to certainconditions (such as when he is double teamed, when he drives toward thelane, when a defender attempts to steal the ball, etc.) and then predictthe ball movements and trajectories based on such learned tendencieswhen the ball is occluded from view during similar conditions.

The ball path logic 1591 can analyze the current data and make adetermination regarding the expected movement of the ball based on thecurrent conditions associated with the person dribbling the ball. Forexample, if the person dribbling the ball is being trapped by twodefenders (the two defender scenario), the ball path logic 1591 candetermine that it will be unlikely for the person dribbling the ball touse a behind-the-back dribble (or other dribble type) in the directionof one of the defenders and determine the available directions that theball could be dribbled to approximate a likely movement of the ball. Theball path logic 1591 can then evaluated the approximated movement of theball as described above.

If the ball path logic 1591 is not able to approximate the movement ofthe ball from the currently available data, the ball path logic 1591 mayable to approximate the movement of the ball based on historical data1596 associated with the person dribbling the ball. In other words, theball path logic 1591 can determine an approximate movement of the ballbased on the previous movements of the person in similar situations. Forexample, in the two defender scenario, the ball path logic 1591 maydetermine based on historical data 1596 that the person dribbling theball usually attempts to dribble between the defenders when confrontedby two defenders. Using that determination, the ball path logic 1591 canapproximate a trajectory or movement of the ball that has the balltravelling between the defenders. The ball path logic 1591 can thenevaluate the approximated movement of the ball as described above.

In one embodiment, the ball path logic 1591 can determine the currentposition and situation of the person dribbling the ball and determinethe possible moves that can be made from that position and situation.The ball path logic 1591 can then determine the probabilities of theperson dribbling the ball executing each of the possible moves and usethe probability determinations in determining the approximate movementof the ball. For example, in the two defender scenario, there aremultiple moves or sequences possible for the person dribbling the ballsuch as: pick up the ball; cross-over dribble from left to right;cross-over dribble from right to left; front to back between-the-legsdribble from left to right; front to back between-the-legs dribble fromright to left; back to front between-the-legs dribble from right toleft; back to front between-the-legs dribble from left to right;behind-the-back dribble from left to right; and behind-the-back dribblefrom right to left. However, the person dribbling the ball, based ontheir historical data 1596, may only be capable of doing a few of thepossible dribble sequences and may not have the requisite skill leveland/or may not have used the other possible dribble sequences in thepast. The ball path logic 1591 can assign a higher probability to thedribbling sequences previously performed by the person and a lowerprobability to the other dribbling sequences. In contrast, a moreskilled player be able to perform most or all of the possible dribblingsequences and the ball control logic 1591 would assign differentprobabilities to the possible sequences. The ball path logic 1591 canthen use the assigned probabilities to determine an approximate movementfor the ball. The ball path logic 1591 can then evaluate theapproximated movement of the ball for accuracy as described above.

In one embodiment, the ball path logic 1591 can processvideo/audio/depth sensing/motion sensing sequences that include taggeddescriptors provided by a reviewer of the camera data 1578 whichdescribe dribble patterns, dribbler patterns and/or levels of dribblingcapability in a quantitative or qualitative way. The ball path logic1591 can use the tagged descriptors in building a knowledge base formachine learning and/or artificial intelligence. The degree of taggingprovided in the video/audio/depth sensing/motion sensing data can varybetween no tagging, light tagging, or full tagging. As the knowledgebase for the ball path logic 1591 increases, the machine learning and/orartificial intelligence of the ball path logic 1591 can be used to“track” the movements of the ball and the person dribbling the ball forlonger periods when the ball and the person are mostly occluded from theview of cameras 1502 or sensors 1514.

In another embodiment, the ball path logic 1591 may be able to use datafrom only a single sensor (e.g., a camera 1502, audio detector, depthsensor or motion sensor) to accomplish ball movement determinationsacross an entire athletic playing surface even though the ball and/oraspects of the person dribbling the ball are occluded much of the time.The ball path logic 1591 can use one or more of the techniques describedabove to determine the movement of the ball with only an occasionaldetection of the ball by identification logic 1592 to locate/relocatethe ball between analysis techniques.

In still another embodiment, the ball path logic 1591 can use machinelearning and/or artificial intelligence to analyze the historical data1596 to uncover patterns and trend information. The ball path logic 1591can then use the pattern and trend information when determining theprobabilities associated with the location and movement of the ball.

The defender motion logic 1593 can be used to identify the specificperson guarding the person dribbling the ball and determine orapproximate the movements and actions of the identified defender. Thedefender motion logic 1593 can determine the movements and actions ofone or more defenders (once identified) guarding a person dribbling theball even if the defender(s) cannot be continuously identified byidentification logic 1592 from the camera data 1578. For example, theidentification logic 1592 (or the computer vision logic 1564) may not beable to identify a defender because the defender may not be present incamera data 1578. The defender may not be present in some portions ofthe camera data 1578 due to the defender being concealed from the fieldof view of the cameras 1502 by the person dribbling the ball and/or oneor more other persons on the athletic playing surface. Further, even ifthe defender is present in the camera data 1578, the identificationlogic 1592 may not be able to identify the defender because the defenderis obscured in the camera data 1578 due to poor lighting conditionsand/or partial occlusion of the defender (particularly those featuresused to identify the defender).

Prior to determining the movements of a defender, the defender motionlogic 1593 may determine whether a defensive player is defending theperson dribbling the ball. The defender motion logic 1593 can determinewhether a defensive player or players is guarding the person dribblingthe ball based on the distance between the defensive player and theperson dribbling the ball and the position and/or orientation of thedefensive player with respect to the person dribbling the ball. Forexample, a defensive player within 5 feet of the person dribbling theball and facing the person dribbling the ball can be considered to beguarding the person dribbling the ball. Once the defender motion logic1593 has determined a defensive player is a defender of the persondribbling the ball, the defender motion logic 1593 can identify thespecific defender using information from the identification logic 1592regarding the identity of the player. The defender motion logic 1593 canuse identification information directly from the identification logic1592 or the computer vision logic 1564 to specifically identify thedefender. In another embodiment, the defender motion logic 1593 canperform the identification of the specific defender based on theinformation from the identification logic 1592. For example, thedefender motion logic may use body motion data 1598 to identify thespecific defender since each player can have a unique body motionprofile. The defender motion logic 1593 can then designate and storespecific movements and actions of the specific defender in responding toactions of the person dribbling the ball. The measurement logic 1597 canuse the stored information by the defender motion logic in evaluatingthe performance of the defender.

In one embodiment, the defender motion logic 1593 can identify thelocation in a 3-D space of the defender's fingers, hands, elbows,shoulders, chest, head, waist, back, thighs, knees, calves, hips,ankles, feet, and/or other body parts. In addition, once the individualbody parts have been identified, the defender motion logic 1593 candetermine relative locations of the identified body parts to each other.The defender motion logic 1593 can provide the information of thedefender's body to the body motion data 1598 for use by the objecttracker 1562. For example, the balance logic 1599 can use the bodymotion data 1598 to measure or infer the balance of the defender and thedefender's ability to respond. In one embodiment, the defender's balancecould be relative to the balance of a normal person from a chosen groupor could be relative to the “normal” balance of the particular defenderusing historical data 1596. In another embodiment, since the players onthe athletic playing surface alternate between offense and defense, thedefender motion logic 1593 can specifically identify each of the playersand store corresponding information for each of the players.

The defender motion logic 1593 can use the information from theidentification logic 1592 to determine the movement and/or actions ofthe defender. In addition, the defender motion logic 1593 can assignparameters to the defender movements and/or actions and categorize theoutcome of the particular defensive movements and/or actions. Someexamples of categories that can be used are: the defender committed afoul, which can include information on the foul type and otherparameters associated with the foul; the defender stole the ball fromthe person dribbling the ball (a steal), which can include informationon the action that resulted in the steal and other parameters associatedwith the steal; the defender maintained defensive position (e.g., thedefender is facing the person dribbling the ball and is located at aposition between the person and the basketball goal) on the persondribbling the ball; the defender did not maintain defensive position onthe person dribbling the ball; or other activity outcome descriptors.

When the identification logic 1592 cannot provide specific informationon the location of the defender, possibly due to occlusion or lighting,the defender motion logic 1593 can determine an expected movement of thedefender based on the last known location of the defender fromidentification logic 1592 and other information stored in memory 1566.The defender motion logic 1593 can use computer learning to establishthe most likely movements and/or actions the defender would performbased on any other current data that is available (e.g., data extractedfrom camera data 1578 or data from sensor data 1580 such as depthsensor, motion sensor/accelerometer or sound information) or fromhistorical data 1596 that includes information on what the person ismost likely to do in a particular situation or environment.

The defender motion logic 1593 can analyze the current data and make adetermination regarding the expected movement and/or actions of thedefender based on the current conditions associated with the defender.For example, if the defender is guarding a person dribbling to thedefender's right and the defender has been previously moving to theright, the defender motion logic 1593 can determine that the defendermay continue to move to the right and that it is unlikely for thedefender to slide to the left. Thus, the defender motion logic 1593 canuse computer learning and/or artificial intelligence to determine thepossible directions the defender could move, look at the outcome of thedefensive sequence based on the location of the defender fromidentification logic 1592 and determine which defensive movement wasactually used.

If the defender motion logic 1593 is not able to approximate accuratelythe movement and/or actions of the defender from the currently availabledata, the defender motion logic 1593 may be able to approximate themovement and/or actions of the defender based on historical data 1596associated with the defender. In other words, the defender motion logic1593 can determine an approximate movement and/or action for thedefender based on the previous movements of the person in similarsituations. For example, when the person dribbling the ball is moving tothe defender's left, the defender motion logic 1593 may determine basedon historical data 1596 that the defender will likely take a step backand then move to the left. Using that determination, the defender motionlogic 1593 can approximate the movement and/or actions of the defenderas backwards and then to the left. The defender motion logic 1593 canthen evaluate the approximated movement of the defender once informationon the defender becomes available from the identification logic 1592.

In one embodiment, the defender motion logic 1593 can determine thecurrent position and situation of the defender relative to the persondribbling the ball and determine the possible moves and/or actions thatcan be made from that position and situation. The defender motion logic1593 can then determine the probabilities of the defender executing eachof the possible moves and/or actions and use the probabilitydeterminations in predicting or otherwise estimating the approximatemoves or actions of the defender. For example, if the defender isguarding a person making a cross-over dribble from right to left (asseen by the defender), there are a number of motions and/or actions thedefender could make such as: a low lunge forward to steal the ball withtwo hands; a low lunge forward with the right hand to tip the ball up; alow lunge forward with the right hand to tip the ball sideways; a slideto the left maintaining distance with the person dribbling the ball; astep back to allow more space and prevent the person dribbling the ballfrom advancing to the hoop; a jump forward to block the vision of theperson and prevent a pass or shot; a jump left or right or up to preventa pass by the person; or a “broken ankles” stumble because the dribblemove was so effective that the defender lost their defensive positionand/or balance. However, the defender, based on their historical data1596, may only be capable of doing a few of the possible movements oractions and may not have the requisite skill level and/or may not haveused the other possible movements or actions in the past. The defendermotion logic 1593 can assign a higher probability to the movementsand/or actions previously performed by the person and a lowerprobability to the other movements and/or actions. In contrast, a moreskilled player may be able to perform most or all of the possiblemovements and/or actions and the defender motion logic 1593 would assigndifferent probabilities to the possible sequences. The defender motionlogic 1593 can then use the assigned probabilities to predict anapproximate movement and/or action for the defender. The defender motionlogic 1593 can then evaluate the approximated movement of the defenderonce information on the defender becomes available from theidentification logic 1592 to determine whether the prediction isaccurate.

In one embodiment, the defender motion logic 1593 can process varioussequences, e.g., video sequences, audio sequences, depth sensorsequences or motion sensor sequences, about a defender that includestags (or tagged descriptors) with information about defender patternsand/or defender capabilities in a quantitative or qualitative way. Thetags provide information and/or a description about the content (e.g.,the actions of the defender) of a sequence and can be associated withthe sequence (or file) similar to metadata. A sequence can have a singletag describing the actions of the defender or multiple tags describingdifferent actions of the defender. The tags can correspond to actions orcategories of actions, e.g., a steal or block, which are recognized bythe defender motion logic 1593. A user can review the sequences (whichmay be obtained from camera data 1578) and apply the appropriate tag(s)to the defender's actions in the sequence. When applying a tag, the usercan select from a predetermined list of tags and/or can create their owntag. The degree of tagging provided in the sequence data can varybetween no tagging, light tagging, or heavy tagging. The defender motionlogic 1593 can use the tagged descriptors in building a knowledge basefor machine learning and/or artificial intelligence. As the knowledgebase for the defender motion logic 1593 increases, the machine learningand/or artificial intelligence of the defender motion logic 1593 can beused to “track” the movements of the defenders for longer periods whenthe defenders are mostly occluded from the view of cameras 1502 orsensors 1514.

In another embodiment, the defender motion logic 1593 may be able to usedata from only a single sensor (e.g., a camera 1502, audio detector,depth sensor or motion sensor) to accomplish defender movement and/oraction determinations across an entire athletic playing surface eventhough the defender may be occluded much of the time. The defendermotion logic 1593 can use one or more of the techniques described aboveto determine the movements of the defender with only an occasionaldetection of the defender by identification logic 1592 toassess/reassess the position of the defender between analysistechniques.

The measurement logic 1597 can be used to analyze data about the persondribbling the ball and the defender of the person dribbling the ball.The measurement logic can use information from identification logic1592, ball path logic 1591, defender motion logic 1593, balance logic1599, historical data 1596, body motion data 1598 and/or evaluation data1582 to analyze the performance and capabilities of the person dribblingthe ball and the defender(s) of the person dribbling the ball.

The measurement logic 1597 can determine the proficiency of the persondribbling the ball with respect to many different dribblingcharacteristics. For example, some of the dribbling characteristics ofthe person dribbling the ball that can be evaluated by the measurementlogic 1597 can include the person dribbling the ball performing very lowdribbling, very fast dribbling, fast changes in dribbling speed (i.e.,acceleration or deceleration), fast changes in dribbling direction,multiple fast changes in dribbling direction, stopping a forward orsideways motion very quickly while maintaining a dribble, fasttransitions from dribbling to shot release, fast transitions fromdribbling to pass (for a wide variety of passing types and situations),and/or any other desired dribbling characteristic. Each of thesedribbling characteristics can be described by one or more quantitativeparameters. For example, very low dribbling can be characterized bymaintaining the dribble height (actual or average) under a predefinedvalue, very fast dribbling can be characterized by the personmaintaining the dribbles per second above a predefined value, fastchanges in dribbling speed can be characterized by completing a changein the dribbles per second within a predefined time period, fast changesin dribbling direction can be characterized by completing a change ofdirection within a predefined time period, multiple fast changes indribbling direction can be characterized by completing several changesof direction within a predefined time period, stopping a forward orsideways motion very quickly while maintaining the dribble can becharacterized by ending an active motion (while maintaining a dribblingmotion) within a predefined time period and/or predefined distance, fasttransitions from dribbling to shot release can be characterized by thetime to transition from a dribbling motion to a shooting motion beingwithin a predefined time period, and fast transitions from dribbling topass can be characterized by the time to transition from a dribblingmotion to a passing motion being within a predefined time period. Eachof the dribbling characteristics can further be characterized by sometype of qualitative or quantitative score from scoring logic 1595 thatindicates the level of skill required to achieve proficiency over thedribbling characteristic. In one embodiment, the measurement logic 1597can provide the person's dribbling characteristics relative toindividual defensive players.

The measurement logic 1597 can also determine the proficiency of theperson dribbling the ball with respect to the person's ability toachieve the same pattern of dribbling every time. The measurement logic1597 can evaluate a person's ability to complete training sequences thatcan require specified dribble speeds, specified dribble heights,specified changes in speed, specified changes in dribble position,specified changes inhead/shoulder/elbow/hand/finger/chest/waste/thigh/knee/ankle/footposition and/or specified maintenance of balance. The measurement logic1597 can also determine the proficiency of the person dribbling the ballby evaluating whether the person is able to repeat the same dribble movein a highly effective way in game situations. Each of these situationscan be described by a quantitative parameter or set of parameters. Forexample, to evaluate the person's proficiency in completing a trainingsequence, the measurement logic 1597 may individually evaluate theperson's completion of each of the individual tasks (which cancorrespond to one or more parameters) in the training sequence. Each ofthese parameters can further be characterized by some type ofqualitative or quantitative score from scoring logic 1595 that indicatesthe level of skill required to achieve proficiency over the dribblingmoves.

The measurement logic 1597 can evaluate the performance of the persondribbling the ball based on the number and type of different movementsthat are performed by the person dribbling the ball in response to thesame or similar situation. In other words, the measurement logic 1597can determine the person's proficiency in not repeating the same patternof dribbling every time. The ability of the person dribbling the ball tovary the dribbling motions used in response to particular situations canbe used to limit the effectiveness of the defensive player inidentifying and responding to repetitive patterns in the person'sdribbling motion. The measure of the person's ability not to repeat thesame pattern of dribbling can be described by one or more quantitativeparameters. Each of these parameters can further be characterized bysome type of qualitative or quantitative score from scoring logic 1595that indicates the level of skill required to achieve thatnon-repetitive capability.

In one embodiment, the measurement logic 1597 can determine the numberof assists by the person dribbling the ball. An assist can be defined asthe last pass to a person that directly leads to the scoring of abasket. In addition, for an assist, the person receiving the pass mustmove directly toward the basket in a “scoring motion,” which may includedribbling the ball. The measurement logic 1597 can be used to determinewhen the person dribbling the ball makes a pass to a teammate and whenthe teammate receiving the pass takes (and makes) a shot at the basket(i.e., scores a field goal). The measurement logic 1597 can track themotions and actions of the teammate receiving the pass and determine ifthe teammate receiving the pass has performed a “scoring motion.” Themeasurement logic 1597 can determine a scoring motion based on themovements and actions of the teammate involved with the scoring of abasket and numerous other factors such as the amount of time betweenreceiving the pass and scoring the basket, the movement of the teammatetowards the basket and the location where the shot was taken relative tothe location where the pass was caught. The measurement logic 1597 canalso track the passer's number of assists for each of his/her teammates.In another embodiment, the measurement logic 1597 can determine whetherthe person dribbling the ball has taken (and made) a shot at the basket.

The measurement logic 1597 can also evaluate the effectiveness of theperson dribbling the ball with respect to the defender(s) guarding theperson. The measurement logic 1597 can use information from balancelogic 1599 to determine changes in the body orientation and position andbalance of the defender as a result of a dribbling move. For example, aperson dribbling the ball may cause a defender to stumble and/or fallafter performing a particular dribbling move, e.g., a cross-overdribble, which enables the person dribbling the ball to “defeat” thedefender and progress to the basket or an open position on the court.The measure of the ability of the person dribbling the ball tonegatively impact a defender's balance and position and orientation toenable the person dribbling the ball to advance to the basket can bedescribed by one or more quantitative parameters. Each of theseparameters can further be characterized by some type of qualitative orquantitative score from scoring logic 1595 that indicates the level ofability in disrupting the defender's body orientation and position andbalance.

The measurement logic 1597 can also determine the ability of the persondribbling the ball to complete one or more related objectives such as ahigh number of assists and/or a low number of turnovers. The objectivescan be calculated based on the person's overall performance or withrespect to individual defensive players. The measure of a dribbler'sability to achieve the related objectives and to determine how much ofthe achievement of the higher-order objectives is due to dribblingexpertise can be described by a quantitative parameter or set ofparameters. For example, the measurement logic 1597 can determine theeffectiveness of a dribbling move in generating an open passing lane(that results in a score by the person receiving the ball) for theperson dribbling the ball when evaluating the number of assists. Each ofthese parameters can further be characterized by some type ofqualitative or quantitative score from scoring logic 1595 that indicatesthe level of skill required to achieve the higher order objectives.

The measurement logic 1597 can determine the proficiency of the defenderwith respect to many different defensive characteristics. For example,some of the defensive characteristics of the defender that can beevaluated by the measurement logic 1597 can include the defender havinga very fast forward speed, very fast forward acceleration, very fastforward lunge acceleration, very low forward lunge, very fast sidedefensive speed, very fast side defensive acceleration, very low sidedefensive position, very fast change of direction of side movement, veryfast ending of side movement, very fast reverse speed, very fasttransition from dribbling defensive position to pass interceptionposition, very fast transition from dribbling defensive position to shotdefensive position, and/or any other desired defensive characteristic.Each of these defensive characteristics can be described by one or morequantitative parameters. For example, very fast forward speed can becharacterized by maintaining a forward speed (actual or average) above apredefined value, very fast forward acceleration can be characterized byhaving an acceleration rate above a predefined value, very fast forwardlunge acceleration can be characterized by having a lunge accelerationrate above a predefined value, very low forward lunge can becharacterized by maintaining the defender's forward lunge position belowa predefined height, very fast side defensive speed can be characterizedby maintaining a lateral speed, e.g., a speed in moving to the side,above a predefined value, very fast side defensive acceleration can becharacterized by having a side acceleration rate above a predefinedvalue, very low side defensive position can be characterized bymaintaining a side defensive position below a predefined height, veryfast change of direction of side movement can be characterized byswitching from one side movement to an opposed sided movement within apredefined time period, very fast ending of side movement can becharacterized by stopping a side movement within a predefined time orpredefined distance, very fast reverse speed can be characterized bymaintaining a reverse speed (actual or average) above a predefinedvalue, very fast transition from dribbling defensive position to passinterception position can be characterized by the time to transitionfrom a dribbling defensive position to a passing defensive positionbeing within a predefined time period, very fast transition fromdribbling defensive position to shot defensive position can becharacterized by the time to transition from a dribbling defensiveposition to a shot defensive position being within a predefined timeperiod. Each of the defensive characteristics can further becharacterized by some type of qualitative or quantitative score fromscoring logic 1595 that indicates the level of skill required to achieveproficiency over the defensive characteristic. In one embodiment, themeasurement logic 1597 can provide the defender's defensivecharacteristics relative to individual offensive players (e.g., aspecific person dribbling the ball).

The measurement logic 1597 can also determine the proficiency of thedefender with respect to the defender's ability to achieve one or morepatterns of defensive movements. The measurement logic 1597 can evaluatea defender's ability to complete training sequences that can requirespecified forward speeds, specified defensive heights, specified changesin speed, specified changes in defensive position, specified changes inbody position and/or specified maintenance of balance. The measurementlogic 1597 can also determine the proficiency of the defender byevaluating whether the person is able to repeat the same defensivemovements or actions in a highly effective way in game situations. Eachof these situations can be described by a quantitative parameter or setof parameters. For example, to evaluate the defender's proficiency incompleting a training sequence, the measurement logic 1597 mayindividually evaluate the defender's completion of each of theindividual tasks (which can correspond to one or more parameters) in thetraining sequence. Each of these parameters can further be characterizedby some type of qualitative or quantitative score from scoring logic1595 that indicates the level of skill required to achieve proficiencyover the defensive movements and/or actions.

The measurement logic 1597 can evaluate the performance of the defenderbased on the number and type of different movements that are performedby the defender in response to the same or similar situation. In otherwords, the measurement logic 1597 can determine the defender'sproficiency in not repeating the same defensive movements and/or actionsfor a given situation. The ability of the defender to vary the defensivemovements and/or actions used in response to particular situations canbe used to limit the effectiveness of the person dribbling the ball inidentifying and responding to repetitive patterns in the defender'sdefensive movements. The measure of the person's ability not to repeatthe same pattern of defensive movements can be described by one or morequantitative parameters. Each of these parameters can further becharacterized by some type of qualitative or quantitative score fromscoring logic 1595 that indicates the level of skill required to achievethat non-repetitive capability.

The measurement logic 1597 can also determine the ability of thedefender to complete one or more related objectives such as a such as ahigh number of steals, high number of blocks, high number of deflectedpasses, high number of deflected shots, and/or high number of traps. Theobjectives can be calculated based on the defender's overall performanceor with respect to individual offensive players. The measure of adefender's ability to achieve the related objectives and to determinehow much of the achievement of the related objectives is due todefensive expertise can be described by a quantitative parameter or setof parameters. For example, the measurement data 1597 can determine theeffectiveness of a defensive move in positioning the defender to eitherdeflect a pass from the person dribbling the ball or steal the pass fromthe person dribbling the ball. Each of these parameters can further becharacterized by a qualitative or quantitative score from scoring logic1595 that indicates the level of skill required to achieve the relatedobjectives.

In one embodiment, the measurement logic 1597 can determine the use of a“screen” on the defender of the person dribbling the ball and thedefender's response to the screen. A screen is a known basketball termthat generally refers to a play or situation when an offensive player,referred to hereafter as “offensive screener,” without the ballestablishes a stationary position to block the path of the defender ofanother offensive player, referred to hereafter as the “offensive screentarget,” moving towards the screener who is setting the screen. Theoffensive screen target can either have the ball or can be attempting toreceive a pass from the person with the ball. The measurement logic 1597can detect the occurrence of a screen by determining the establishmentof a stationary position by the offensive screener near the defender ofthe offensive screen target such that the path of the defenderintersects with the stationary position of the offensive screener.

Note that there are various factors that can be used in thedetermination of whether a screen has occurred. As an example, theprobability of a screen may be increased if the defender contacts theoffensive screener within a predefined time after establishment of thescreener's stationary position. In addition, the orientation of theoffensive screener to the defender of the offensive screen target may beindicative of whether a screen has occurred. In this regard, anoffensive screener often faces the defender when setting a screen so asto help increase the width of the screen and, thus, the screen'seffectiveness of disrupting the path of the defender. In addition, theproximity of the offensive screen target relative to the offensivescreener may indicate whether a screen is occurring. In this regard, anoffensive screen target often passes within a close proximity or evencontacts the offensive screener when passing the offensive screener.Thus, detecting that the offensive screen target has passed theoffensive screener within a predefined distance may indicate theoccurrence of a screen. The measurement logic 1597 may detect any of theevents described above as indicative of a screen and may detect theoccurrence of a screen based on any combination of such factors. As anexample, in assessing whether a stationary position of the offensivescreener constitutes a screen, the measurement logic 1597 may calculatea screen score that is increased by a certain amount for each detectionof an event that indicates the occurrence of the screen while theoffensive screener is in the stationary position. If the screen scoreexceeds a predefined threshold, then the measurement logic 1597 maydetect the occurrence of a screen. In other embodiments, othertechniques for detecting an occurrence of a screen are possible.

When a screen is detected, the measurement logic 1597 may assess howvarious players performed during the screen and track the results overtime to determine a score, referred to herein as “screen score,”indicating each player's skill level in executing screens or defendingagainst screens. As an example, the measurement logic 1597 can determinehow the defensive player responded to the screen. The measurement logic1597 can determine if the defensive player goes “above” or “below” thescreen, stops motion or switches defensive assignment with anotherdefensive player such that the defensive player is no longer guardingthe person dribbling the ball.

In this regard, as is commonly understood in basketball, it is generallydesired for a defensive player to defend against a screen by going“above” the screen. Going “above” the screen generally refers to whenthe defender passes the offensive screener on the same side as theoffensive screen target. This is often the more challenging course ofaction for the defender as it is often difficult to “fight through” thescreen in order to stay on the same side of the offensive screener asthe offensive screen target. However, going “above” the screen oftenallows the defender to maintain a good defensive position with respectto the offensive screen target by staying close to the offensive screentarget through the screen. In contrast going “below” the screengenerally refers to when the defender passes the offensive screener onthe opposite side of the offensive screener as the offensive screentarget. This is often simpler for the defender to achieve, relative togoing “above” the screen, but it results in separation between thedefender and the offensive screen target, which is undesirable as itoften gives the offensive screen target an opportunity to make a playsuch as taking an undefended shot on the goal or driving toward thegoal.

The measurement logic 1597 can determine if the defensive player goesabove or below the screen by determining the position of the defensiveplayer relative to the offensive screener and the offensive screentarget. For example, based on the images captured by the cameras orotherwise, the measurement logic 1597 may determine whether the defenderand the offensive screen target pass on the same side of the offensivescreener. If so, the measurement logic 1597 determines that the defenderhas gone “above” the screen. If the measurement logic 1597 determinesthat the defender and the offensive screen target pass on opposite sidesof the offensive screener, then the measurement logic 1597 determinesthat the defender has gone “below” the screen.

The measurement logic 1597 can track how the defender responds to beingscreened over time and can also track how the defensive player respondsto screens from individual offensive players. As an example, themeasurement logic 1597 may track the number of times that the defendergoes “above” screens during a given time period and provide a parameterindicative of such number (e.g., a percentage of screens that thedefender goes “above” the screen). The measurement logic 1597 maysimilarly track other outcomes, such as the number of times that thedefender goes “below” the screen or otherwise defends against thescreen. The measurement logic 1597 can also determine various parametersindicative of the effectiveness of the defender's responses to screens.As an example, for each screen, the measurement logic 1597 may determinewhether the defender was able to maintain a defensive position withrespect to the offensive screen target (e.g., stayed within a certaindistance of the offensive screen target and/or stayed between theoffensive screen target and the goal) or whether the offensive screentarget was able to perform a certain action coming off of the screen(e.g., within a predefined time period after passing the offensivescreener), such as taking an open shot at the goal or driving toward thegoal unguarded by the defender. The measurement logic 1597 may track thenumber of times that one or more outcomes occur over a given time periodand provide a parameter indicative of such number (e.g., a percentage ofscreens that a certain outcome occurs). The parameters tracked by themeasurement logic 1597 may be correlated with the offensive players sothat the defender's play against the offensive players can beascertained and assessed. As an example, the data provided by themeasurement logic 1597 may be used to determine how many times thedefender went “over” a screen (or performed some other action) set by aparticular screener relative to the number of times that he went “under”such a screen. Thus, the performance of the defender against screens setby the particular screener can be assessed.

The measurement logic 1597 can similarly track the motions and actionsof the offensive players involved with the screen. In this regard, thesame or similar actions and events tracked by the measurement logic 1597for assessing the defender's play can be used to assess the play of theoffensive players. As an example, the measurement logic 1597 may trackthe number of times that the offensive screen target caused his defenderto go “below” the screen or perform some other action during the screen.The measurement logic 1597 can also track the number of times that theoffensive screen target is able to take a shot at the basket, dribbletowards the basket, make a pass to another offensive player, possiblyresulting in an assist, or perform some other action coming off of thescreen.

The measurement logic 1597 can similarly evaluate the performance of theoffensive screener. As an example, the measurement logic 1597 candetermine the proximity of the offensive screener to the defender of theoffensive screen target. The measurement logic 1597 can determine thespeed with which the offensive screener is able to set the screen, i.e.,how fast does the screener obtains a stationary position, and the timingwith which the screener sets the screen, i.e., the time between when thescreener establishes the screen and the time when the defender arrivesat the screener or contacts the screener. The measurement logic 1597 canevaluate the general effectiveness of the offensive screener by trackingone or both of the responses of the offensive screen target and thedefender and can also evaluate the effectiveness of the screener withrespect to individual teammates and/or individual defenders. Inparticular, similar to the tracking of the defender described above, themeasurement logic 1597 can correlate the parameters tracked for a givenoffensive player to the screen defenders in order to track the offensiveplayer's performance against certain defenders. As an example, the dataprovided by the measurement logic 1597 may be used to determine how manytimes a particular offensive screener, offensive screen target, or apair of offensive screener and offensive screen target caused aparticular defender to go “under” a screen (or perform some otheraction).

In one embodiment, the measurement logic 1597 can evaluate theeffectiveness of the offensive screener by determining when the screenerperformed illegal screens. The measurement logic 1597 can determine anillegal screen, sometimes referred to as a “moving” screen, based onwhether the person was called for a foul. The measurement logic 1597 canalso determine illegal screens by evaluating the movements of theoffensive screener regardless of whether a foul is called. Themeasurement logic 1597 can determine an illegal screen by determining anextent to which the screener's hips or torso moves during the screen.The measurement logic 1597 can also detect an illegal screen if thescreener moves (e.g., “sticks out”) a hip, knee, leg, elbow, etc., whilein the stationary position to impede the progress of the defender in animpermissible way.

In one embodiment, the measurement logic 1597 can use entropy modelingto determine when dribbling unpredictability, screening unpredictabilityand/or defender unpredictability is beneficial or detrimental to theplayer and/or team. For example, the measurement logic 1597 maydetermine that dribbling unpredictability in a player is beneficialbecause the unpredictability of the player can make it more difficultfor a defensive player to “scout” the movements of the player. However,the measurement logic 1597 may determine that the dribblingunpredictability of a player is detrimental if the player does not havegood control of the ball and generates a high number of turnovers or lownumber of scores or assists.

The improvement logic 1594 can be used to analyze data about the persondribbling the ball and the defender of the person dribbling the ball andrecommend methods to improve either dribbling capability or defensivecapability and to predict the amount of improvement in dribblingcapability or defensive capability. The improvement logic 1594 can useinformation from identification logic 1592, ball path logic 1591,defender motion logic 1593, balance logic 1599, measurement logic 1597,historical data 1596, body motion data 1598 and/or evaluation data 1582to identify opportunities to improve the performance and capabilities ofthe person dribbling the ball and the defender(s) of the persondribbling the ball.

The improvement logic 1594 can recommend particular practice routines,game drills and technique modifications depending on the specificperformance area of requiring improvement. For example, if themeasurement logic 1597 indicates that a person is dribbling the ball toohigh such that the ball is being stolen frequently by the defender, theimprovement logic 1594 can recommend one or more training or practiceroutines that require the person to dribble with a lower dribblingheight. In another example, if the measurement logic 1597 indicates thata defender often allows a dribbler to easily move past them on the wayto the basket, the improvement logic 1594 can recommend one or moretraining or practice routines that function to improve lateral defensivespeed.

The improvement logic 1594 can map specific training or practiceroutines to performance areas. The improvement logic 1594 can also mapskill level designations, e.g., needs significant improvement, to thetraining or practice routines. Then, when the improvement logic 1594identifies a performance area that requires improvement, the improvementlogic 1594 can select a training or practice routine that has beenmapped to the performance area requiring improvement. The improvementlogic 1594 can also narrow the selection of the training or practiceroutine from the mapping based on the assessment of the performance areaby measurement logic 1597 such that the selected training or practiceroutine better matches the actual skill level of the person.

In another embodiment, the improvement logic 1594 may not be able torecommend a specific practice routine, game drill and techniquemodification because there may be multiple areas requiring improvedperformance and/or there may be multiple practice routines, game drillsand technique modifications that can be used to address a specific arearequiring improvement. For example, if a dribbler often has their shotblocked or tipped when taking a jump shot off the dribble, the problemmight be an inability to get sufficient separation from the defenderprior to picking up the dribble, or the problem might be a slowtransition from dribble to shooting position, or the problem might be aslow shot release, or the problem might be a low shot release (asdetermined by measurement logic 1597 based on the trajectory of the ballfor the shot and the location of the shooter's body parts such ashis/her hand and/or elbow), or the problem might be a combination of theabove challenges. In another example, if a defender is often unable todisrupt a jump shot off the dribble, e.g., block or tip the jump shot,the problem might be an inability to limit the separation by the persondribbling the ball prior to the jump shot, or the problem might be aslow transition from dribbling defensive position to shooting defensiveposition, or the problem might be a off-center hand placement, or theproblem might be a combination of the above challenges.

In situations where there is not a readily identifiable improvementregimen or multiple possible improvement regimens, the improvement logic1594 can select other players, e.g., other dribblers or defenders, whohave previously shown similar dribbling or defensive performance with asubsequent improvement in performance (e.g., an improvement above athreshold amount) based on the dribbling or defensive characteristicsdetermined by the measurement logic 1597. The improvement logic 1594 canstore information on the training or practice routines completed by eachperson in historical data 1596. The improvement logic 1594 can alsostore information of the person's performance level after completion ofa training or practice routine and correlate the change in the player'sperformance level to the training or practice routine. The improvementlogic 1594 can review the selected players' practice techniques andimprovement pace from historical data 1596 to determine an optimal setof practice techniques for the person dribbling the ball or the defenderbeing analyzed by the improvement logic 1594.

In another embodiment, the improvement logic 1594 can use informationfrom scoring logic 1595 to determine the performance areas requiring themost improvement. The improvement logic 1594 can review the historicaldata 1596 for other players, e.g., other dribblers or defenders, whohave a similar performance area requiring improvement and theircorresponding practice techniques to determine an optimal set ofpractice techniques and predicted improvement for the area requiringimprovement.

In one embodiment, the historical data 1596 can include a large databaseof many players with many parameter types undergoing many practice andgame dribbling regimens that can all be quantitatively measured. Theimprovement logic 1594 can implement a methodology to maximize theimprovement process in the most efficient way. For example, theimprovement logic 1594 may identify patterns across multiplequantitative dimensions in order to describe the specific problem andthen prescribe the best approach for improvement.

Balance logic 1599 can be used to measure and/or categorize theeffectiveness of the player's balance on the performance of the player.For example, if the person dribbling the ball has good balance, theperson can more effectively move left, right, back, forward, up, down,and with different dribbling techniques at different speeds,accelerations, heights, and angles. In one embodiment, balance logic1599 can use machine learning and/or artificial intelligence to measureand/or categorize a player's balance directly or indirectly.

In one embodiment, the balance logic 1599 may directly assess thebalance of player by determining and analyzing the center of mass of theplayer relative to the player's body. The balance logic 1599 candetermine a player has good balance if the player's center of mass doesnot rapidly change position in response to movements of the player. Thebalance logic 1599 may also make indirect determinations regardingbalance based on factors such as fluidity, rapid acceleration, footplacement and/or sluggishness. For example, if the balance logic 1599determines that a player has motions that are fluid, the balance logic1599 can determine that the player has better balance than a playerwhose motions are less fluid. Similarly, if the balance logic 1599determines that the player has rapid acceleration in one or moremovements, the balance logic 1599 can determine that the player hasbetter balance. The balance logic 1599 may also make a determination ofbalance regarding a player based on the foot placement of the player inresponse to a variety of different situations.

In addition, the balance logic 1599 can also be used to determine thedefender's ability to respond to particular situations. The defender'sability to respond to a situation is dependent on the actions of theperson dribbling the ball. For example, if the person dribbling ball isattempting a shot with the ball (as determined by ball path logic 1591),the balance logic 1599 can determine if a defender is in a low positionor an extended position and thus determine the defender's ability torespond. For example, if the defender is already in an extended positionas determined by balance logic 1599, the the balance logic 1599 maydetermine that defender does not have the desired muscle contractionavailable to respond appropriately to an up movement by the persondribbling the ball. Further, the balance logic 1599 can also determinewhether the defender's ability to respond is limited by the physicalpositions of other defenders or the physical position of the persondribbling the ball.

In one embodiment, historical data 1596 can include data obtained duringa training sequence in a confined training space, e.g., a boundeddribbling area 1516. An example of a confined training space isdescribed by: U.S. patent application Ser. No. 14/874,555, entitled“Systems and Methods for Monitoring Objects in Athletic Playing Spaces”and filed on Oct. 5, 2015, which is incorporated herein by reference.When historical data 1596 is obtained during a training sequence,information pertaining to the movement of the ball in relation to themovement of the person can be more easily obtained since the camera(s)1502 and the sensors 1514 can be placed at appropriate locations toreduce and possibly eliminate any occlusion of the ball or the person.The complete tracking of the ball and the person in the historical data1596 can permit the ball path logic 1591 to more accurately determineprobabilities of movement when the ball or person becomes occluded. Theball path logic 1591 can determine the expected movement of the ballbased on information in historical data 1596 that is similar to thelocation and position of the dribbler when the ball becomes occluded.

FIG. 6 shows an offensive and defensive basketball player on an athleticplaying surface. As can be seen in the embodiment of FIG. 6, cameras1502 can be located at each end of an athletic playing surface 1650 andcan capture the entire athletic playing surface 1650 within the fieldsof view 1652 of the cameras 1502. The cameras 1502 can also capture anoffensive player 1654, the ball 1656 and a defensive player 1658. Thecameras 1502 may be able to capture a lot of information or very littleinformation on the offensive player 1654, the ball 1656 and thedefensive player 1658 depending on their locations on the athleticplaying surface 1650 and their positions with respect to the cameras1502. As discussed above, the ball 1656 may be occluded from the fieldsof view 1652 of the cameras, as shown in FIG. 6, based on the positionsof the offensive player 1654 and the defensive player 1658. The cameras1502 can also capture information on other players (not shown) on theathletic playing surface 1650. The images of the offensive player 1654,the ball 1656 and the defensive player 1658 captured by the cameras 1502can be processed by the object tracker 1562 as discussed above.

In one embodiment, the computing device 1504 can use an augmentedreality system to provide a training sequence that simulates a gamesituation for a person dribbling the ball or a defender. As shown inFIG. 7, an augmented reality system 1710 can include augmented realitylogic 1712. The augmented reality logic 1712 can be implemented insoftware, hardware, firmware or any combination thereof. The augmentedreality logic 1712 can be part of the object tracker 1562 or be astand-alone system in memory 1566. The augmented reality logic 1712 cantransmit, using communication interface 1576, images associated with asimulated game situation to a headset 1714 worn by the user. The headset1714 can have a closed configuration, e.g., a full-sized helmet, thatprevents the user from seeing any of the surrounding physicalenvironment, or the headset 1714 can have an open configuration, e.g., apair of glasses, that permits the user to see the surroundingenvironment while a projection of the game is occurring. In oneembodiment, the headset 1714 can be an output device 1508. The system1500 can capture the user's responses to the simulated game situationwith cameras 1502 and/or sensors 1514 and process the capturedinformation with object tracker 1562 as described above. The objecttracker 1562 can provide information on the user's responses to theaugmented reality logic 1712, which can update the simulation providedto the user.

The augmented reality logic 1712 can match the skill of the simulateddefender to the particular skills to be developed in the trainingsequence. For example, the skill level of the simulated defender may beset lower than the skill level of the person in the training sequence topermit the person dribbling the ball to develop new moves that areeffective against less proficient defenders. In contrast, the skilllevel of the simulated defender can be set greater than the persondribbling the ball to permit the person to increase their skill or tolearn ball protection sequences. The augmented reality logic 1712 canalso be used to simulate multiple defenders during the trainingsequence. In one embodiment, the skill level of the simulated defendercan be based on the skill of an actual defender as represented by thedefender's defensive characteristics collected and stored in historicaldata 1596. In another embodiment, the skill level of the simulateddefender can be based a combination of skills of several differentdefenders or can be generated by the augmented reality logic 1712 toprovide the desired set of skills in the simulated defender.

The augmented reality logic 1712 can also match the skill of thesimulated dribbler to the particular skills to be developed by thedefender in the training sequence. For example, the skill level of thesimulated dribbler may be set lower than the skill level of the defenderin the training sequence to permit the defender to develop new movesthat are effective against less proficient dribblers. In contrast, theskill level of the simulated dribbler can be set greater than thedefender to permit the defender to increase their skill or to learndefensive sequences. The augmented reality logic 1712 can also be usedto simulate multiple offensive players during the training sequence. Inone embodiment, the skill level of the simulated dribbler can be basedon the skill of an actual dribbler as represented by the dribbler'sdribbling characteristics collected and stored in historical data 1596.In another embodiment, the skill level of the simulated dribbler can bebased a combination of skills of several different dribblers or can begenerated by the augmented reality logic 1712 to provide the desired setof skills in the simulated dribbler.

In a further embodiment, the augmented reality logic 1712 can also beused to add simulated court lines or a simulated basketball hoop toenvironments where no court is actually present during the trainingsequence. By providing simulated boundaries and/or the basketball hoopto the headset 1714 of the user when such conditions are not physicallypresent, the augmented reality logic 1712 can provide a more realistictraining environment and increase the benefit of the training sequencefor the user.

The augmented reality logic 1712 can develop the skill level of thesimulated defender or person dribbling the ball in the training sequenceby using machine learning and/or artificial intelligence in conjunctionwith historical data 1596. For example, factors such as the balance,footwork, foot placement, and/or acceleration of the player can be usedin addition to the player's movements, e.g., body part placement and/ororientation, and/or ball motion can be used in establishing the skilllevel of the simulated player(s).

In still another embodiment, the augmented reality system 1710 can beused to allow one or more players to operate in an environment which hasthe appearance of a 10 or more player practice. In a further embodiment,the augmented reality system 1710 can be used by two separate people indifferent locations to simulate a one-on-one contest between the people.One of the persons can be an offensive player and the other person canbe a defensive player. The cameras 1502 can capture information on eachof the players at their respective locations and provide the informationto their corresponding object tracker 1562 which can process theinformation. The object tracker 1562 can then provide the information,e.g., location on the court, player stance, player movements, etc., onthe one player to the augmented reality logic 1712 used by the otherplayer which can then use the information from the object tracker 1562on the one player to simulate the location and movements of the oneplayer in the simulation provided to the other player. For example, adefender can be located at the foul line and in a stationary positionwhile the dribbler can be located at the “top of the key” and movingtowards the defender. The object tracker 1562 in the system 1500 of thedefender can capture the defender's location on the court and thedefensive stance and position of the defender. The information on thedefender can then be provided to the augmented reality system 1710 ofthe dribbler, which can then generate a simulation of the defender atthe defender's location at the foul line and with the defender'sstationary position for the dribbler. Similarly, the object tracker 1562in the system 1500 of the dribbler can capture the dribbler's locationon the court and the dribbling stance and movement of the dribbler. Theinformation on the dribbler can then be provided to the augmentedreality system 1710 of the defender, which can then generate asimulation of the dribbler at the dribbler's location at the top of thekey and with the dribbler's movement toward the defender. Similartechniques may be used with other numbers of players in any number oflocations. As an example, a five-on-five game may be simulated with eachplayer at a different court or other physical location and viewingsimulations of the other nine players.

In one embodiment, as the information capture rate from the cameras 1502and/or sensors 1514 increases, fewer cameras 1502 and/or sensors 1514are required to obtain the same amount of information and/or data forprocessing. For example, a camera 1502 that captures 1000 frames persecond would provide more data to computing device 1504 than a camera1502 that captures 30 frames per second.

In another embodiment, an audio sensor can be used to determine one ormore dribbling characteristics. For example, an audio sensor can detecta change in the sound associated with the dribbling motion and theobject tracker 1562 can use the detected change in sound to determine acorresponding change in a dribbling characteristic such as dribblingrate or speed. In addition, the audio sensor may also be used to assistwith determining the ball trajectory when the ball is occluded fromview. For example, the audio sensor can detect the sound the ball makeswhen striking the floor and provide that information to identificationlogic 1592 and ball path logic 1591 to assist in determining thetrajectory of the ball. The sound of the ball striking the floor can beused with other detected information (such as the detection of the ballleaving a person's hands) to determine the speed of the ball based onthe time difference between the detected information and when the soundof the ball striking the floor is detected. Further, the detection ofthe ball striking the floor in a repetitive pattern may indicate thatthe person is dribbling the ball, even if the ball is occluded from theview of the cameras 1502.

Information passed between the different components in the system may betransmitted using a number of different wired and wireless communicationprotocols. For instance, for wire communication, USB compatible,Firewire compatible and IEEE 1394 compatible hardware communicationinterfaces and communication protocols may be used. For wirelesscommunication, hardware and software compatible with standards such asBluetooth, IEEE 802.11a, IEEE 802.11b, IEEE 802.11x (e.g. other IEEE802.11 standards such as IEEE 802.11c, IEEE 802.11d, IEEE 802.11e,etc.), IrDA, WiFi and HomeRF.

Although the foregoing invention has been described in detail by way ofillustration and example for purposes of clarity and understanding, itwill be recognized that the above described invention may be embodied innumerous other specific variations and embodiments without departingfrom the spirit or essential characteristics of the invention. Certainchanges and modifications may be practiced, and it is understood thatthe invention is not to be limited by the foregoing details, but ratheris to be defined by the scope of the appended claims.

Now, therefore, the following is claimed:
 1. A system for trackingdribbling, comprising: at least one camera configured to capture imagesof a player dribbling an object at the sporting event; memory; at leastone processor configured to receive image data defining the images fromthe at least one camera, the at least one processor configured toidentify the object within the images and to identify a plurality ofdribbles of the object by the player based on the images, wherein the atleast one processor, for each of the plurality of dribbles, isconfigured to identify which hand of the player is used for dribblingthe object and to determine at least one parameter indicative of dribbleperformance for the identified hand, wherein the at least one processoris configured to determine whether a defender is guarding the playerbased on an orientation of the defender relative to the player, andwherein the at least one processor is configured to store the at leastone parameter for each of the plurality of dribbles in the memory and tocalculate, based on the stored at least one parameter and whether thedefender is determined to be guarding the player, at least one valuecharacterizing a dribble performance for one hand of the player duringthe plurality of dribbles; and an output interface configured toprovide, based on the at least one value, an output indicative ofdribble performance of the player for the plurality of dribbles.
 2. Thesystem of claim 1, wherein the at least one processor is configured toestimate a trajectory of the object when the object is occluded in theimages.
 3. The system of claim 1, wherein the output includes the atleast one value characterizing the dribble performance and indicatesthat the at least one value is correlated with the one hand.
 4. A systemfor tracking dribbling, comprising: at least one camera configured tocapture images of a player dribbling a basketball; at least oneprocessor configured to receive image data defining the images from theat least one camera, the at least one processor configured to identifythe basketball within the images and to identify a plurality of dribblesof the basketball by the player based on the images, wherein the atleast one processor, for at least one of the plurality of dribbles, isconfigured to determine at least one parameter indicative of dribbleperformance, wherein the at least one processor is configured todetermine whether a defender is guarding the player based on a distancebetween the player and the defender, and wherein the at least oneprocessor is configured to determine a value characterizing a dribbleperformance of the player based on the at least one parameter andwhether the defender is determined to be guarding the player during theat least one of the plurality of dribbles; and an output interfaceconfigured to provide an output indicative of the dribble performance ofthe player based on the at least one parameter, wherein the outputincludes the value and indicates that the value is correlated with thedefender.
 5. The system of claim 4, wherein the at least one processoris configured to estimate a trajectory of the basketball when thebasketball is occluded in the images.
 6. The system of claim 4, furthercomprising memory, wherein the at least one processor is configured tostore a plurality of parameters in the memory, each of the plurality ofparameters characterizing a dribble performance of the player for arespective one of the plurality of dribbles, wherein the output is basedon each of the plurality of parameters.
 7. The system of claim 4,wherein the output includes a value characterizing a dribble performanceof the player for the plurality of dribbles, and wherein the outputindicates that the value is correlated with at least one defenderguarding the player.
 8. The system of claim 4, further comprisingmemory, wherein the at least one processor is configured to identify thedefender, and wherein the at least one processor is configured to storethe at least one parameter in the memory and to correlate the at leastone parameter with an identifier that identifies the defender.
 9. Thesystem of claim 4, wherein the at least one processor is configured todetermine an orientation of the defender relative to the player based onthe images, and wherein the at least one processor is configured todetermine whether the defender is guarding the player during the atleast one of the plurality of dribbles based on the orientation.
 10. Thesystem of claim 4, wherein the at least one processor is configured toidentify which hand of the player is used for dribbling the basketballduring the at least one of the plurality of dribbles.
 11. The system ofclaim 10, wherein the at least one processor is configured to determine,based on the identified plurality of dribbles, a first parameterindicative of a dribbling performance of the player with his or her lefthand and a second parameter indicative of a dribbling performance of theplayer with his or her right hand.
 12. The system of claim 10, furthercomprising memory, wherein the at least one processor is configured tostore the at least one parameter in the memory and to correlate the atleast one parameter with the identified hand.
 13. The system of claim 4,wherein the at least one processor is configured to identify at leastone body part of the player in the images and to determine a location ofthe body part based on the images, and wherein the at least oneprocessor is configured to determine a dribble type for the at least onedribble based on the location of the at least one body part.
 14. Thesystem of claim 13, further comprising memory, wherein the at least oneprocessor is configured to store the at least one parameter in thememory and to correlate the at least one parameter with the dribbletype.
 15. The system of claim 4, wherein the at least one processor isconfigured to determine an orientation of the defender relative to theplayer based on the images, and wherein the at least one processor isconfigured to determine whether the defender is guarding the playerduring the at least one of the plurality of dribbles based on theorientation.
 16. A system for tracking dribbling, comprising: at leastone camera configured to capture images of a player dribbling abasketball; memory; at least one processor configured to receive imagedata defining the images from the at least one camera, the at least oneprocessor configured to identify the basketball within the images and toidentify a plurality of dribbles of the basketball by the player basedon the images, wherein the at least one processor, for at least one ofthe plurality of dribbles, is configured to determine at least oneparameter indicative of dribble performance, wherein the at least oneprocessor is configured to identify a defender and to determine whetherthe defender is guarding the player, wherein the at least one processoris configured to determine a value characterizing a dribble performanceof the player based on the at least one parameter and whether thedefender is determined to be guarding the player during the at least oneof the plurality of dribbles, and wherein the at least one processor isconfigured to store the at least one parameter in the memory and tocorrelate the at least one parameter with an identifier that identifiesthe defender; and an output interface configured to provide an outputindicative of the value, wherein the output includes the value, andwherein the output indicates that the value is correlated with thedefender.
 17. A system for tracking dribbling event, comprising: atleast one camera configured to capture images of a player dribbling abasketball; at least one processor configured to receive image datadefining the images from the at least one camera, the at least oneprocessor configured to identify the basketball within the images and toidentify a plurality of dribbles of the basketball by the player basedon the images, wherein the at least one processor, for at least one ofthe plurality of dribbles, is configured to determine at least oneparameter indicative of dribble performance and to determine whether adefender is guarding the player based on an orientation of the defenderrelative to the player, wherein the at least one processor is furtherconfigured to determine a value characterizing a dribble performance ofthe player for the plurality of dribbles based on the at least oneparameter and whether the defender is determined to be guarding theplayer; and an output interface configured to provide an outputindicative of the dribble performance of the player based on the atleast one parameter, wherein the output includes the value and indicatesthat the value is correlated with the defender.
 18. A method fortracking dribbling, comprising: capturing images of a player dribbling abasketball with at least one camera; receiving image data defining theimages from the at least one camera; identifying the basketball withinthe images with at least one processor; identifying a plurality ofdribbles of the basketball by the player based on the identifiedbasketball within the images; for the at least one of the identifieddribbles, determining with the at least one processor which hand of theplayer is used for dribbling the basketball; for the at least one of theidentified dribbles, determining with the at least one processor atleast one parameter indicative of dribble performance for one hand ofthe player based on the images; determining with the at least oneprocessor whether a defender is guarding the player based on anorientation of the defender relative to the player; calculating with theat least one processor a value characterizing a dribble performance forthe one hand based on the at least parameter and the determining whetherthe defender is guarding the player; and providing, via an outputinterface, an output indicative of dribble performance of the player forthe plurality of dribbles based on the value.
 19. The method of claim18, further comprising: identifying the basketball in a plurality offrames of the image data; determining a respective location of thebasketball for each of the plurality of frames; and estimating atrajectory of the basketball between the at least two of the pluralityof frames when the basketball is occluded in the images based on thedetermined locations of the basketball for the plurality of frames. 20.The method of claim 18, further comprising: storing, in memory, the atleast one parameter; and correlating the at least one parameter in thememory with an identifier identifying the defender.
 21. The method ofclaim 18, further comprising identifying at least one body part of theplayer in the images with the at least one processor; determining adribble type with the at least one processor based on the trajectory ofthe basketball relative to a location of the at least one body part;storing, in memory, the at least one parameter; and correlating the atleast one parameter in the memory with an identifier identifying thedribble type.
 22. The method of claim 18, further comprisingdetermining, with the at least one processor based on the identifiedplurality of dribbles, a first parameter indicative of a dribblingperformance of the player with his or her left hand and a secondparameter indicative of a dribbling performance of the player with hisor her right hand.
 23. A method for tracking dribbling, comprising:capturing images of a player dribbling a basketball with at least onecamera; receiving image data defining the images from the at least onecamera; identifying the basketball within the images with at least oneprocessor; identifying a plurality of dribbles of the basketball by theplayer based on the identified basketball within the images; for the atleast one of the identified dribbles, determining at least one parameterindicative of dribble performance for the player based on the imageswith the at least one processor; providing, via an output interface, anoutput indicative of the dribble performance based on the at least oneparameter; determining with the at least one processor whether adefender is guarding the player during at least one of the identifieddribbles based on the images, wherein the at least one parameter isbased on the determining whether the defender is guarding the player;storing, in memory, the at least one parameter; and correlating the atleast one parameter in the memory with an identifier identifying thedefender; and determining a distance between the player and the defenderwith the at least one processor based on the images, wherein thedetermining whether the defender is guarding the player is based on thedistance.
 24. A system for tracking dribbling, comprising: at least onecamera configured to capture images of a player dribbling a basketball;at least one processor configured to receive image data defining theimages from the at least one camera, the at least one processorconfigured to identify the basketball within the images and to identifya plurality of dribbles of the basketball by the player based on theimages, wherein the at least one processor is configured to identifywhich hand of the player is used for each of the identified plurality ofdribbles, wherein the at least one processor is configured to determine,based on the identified plurality of dribbles, a first parameterindicative of a dribbling performance of the player with his or her lefthand and a second parameter indicative of a dribbling performance of theplayer with his or her right hand, and wherein the at least oneprocessor is configured to determine whether a defender is guarding theplayer based on an orientation of the defender relative to the player;and an output interface configured to provide an output based on atleast one of the first parameter and the second parameter and based onwhether the defender is determined to be guarding the player.
 25. Amethod for tracking dribbling, comprising: capturing images of a playerdribbling a basketball with at least one camera; receiving image datadefining the images from the at least one camera; identifying thebasketball within the images with at least one processor; identifying aplurality of dribbles of the basketball by the player based on theidentified basketball within the images; for the at least one of theidentified dribbles, determining at least one parameter indicative ofdribble performance for the player based on the images with the at leastone processor; determining with the at least one processor anorientation of a defender relative to the player based on the images;determining with the at least one processor whether the defender isguarding the player during the at least one of the identified dribblesbased on the orientation; determining with the at least one processor avalue indicative of dribble performance of the player based on the atleast one parameter and the determining whether the defender is guardingthe player; and providing an output indicative of the value via anoutput interface.
 26. The method of claim 25, further comprising:determining with the at least one processor a distance between theplayer and the defender based on the images; and wherein the determiningwhether the defender is guarding the player is based on the determiningthe distance.